Skip to main content
Version: v2.6

Pipelines

Notes

  • As of Rancher v2.5, Git-based deployment pipelines are now deprecated. We recommend handling pipelines with Rancher Continuous Delivery powered by Fleet. To get to Fleet in Rancher, click ☰ > Continuous Delivery.

  • Pipelines in Kubernetes 1.21+ are no longer supported.

  • Fleet does not replace Rancher pipelines; the distinction is that Rancher pipelines are now powered by Fleet.

Rancher's pipeline provides a simple CI/CD experience. Use it to automatically checkout code, run builds or scripts, publish Docker images or catalog applications, and deploy the updated software to users.

Setting up a pipeline can help developers deliver new software as quickly and efficiently as possible. Using Rancher, you can integrate with a GitHub repository to setup a continuous integration (CI) pipeline.

After configuring Rancher and GitHub, you can deploy containers running Jenkins to automate a pipeline execution:

  • Build your application from code to image.
  • Validate your builds.
  • Deploy your build images to your cluster.
  • Run unit tests.
  • Run regression tests.
note

Rancher's pipeline provides a simple CI/CD experience, but it does not offer the full power and flexibility of and is not a replacement of enterprise-grade Jenkins or other CI tools your team uses.

Concepts​

For an explanation of concepts and terminology used in this section, refer to this page.

How Pipelines Work​

After enabling the ability to use pipelines in a project, you can configure multiple pipelines in each project. Each pipeline is unique and can be configured independently.

A pipeline is configured off of a group of files that are checked into source code repositories. Users can configure their pipelines either through the Rancher UI or by adding a .rancher-pipeline.yml into the repository.

Before pipelines can be configured, you will need to configure authentication to your version control provider, e.g. GitHub, GitLab, Bitbucket. If you haven't configured a version control provider, you can always use Rancher's example repositories to view some common pipeline deployments.

When you configure a pipeline in one of your projects, a namespace specifically for the pipeline is automatically created. The following components are deployed to it:

  • Jenkins:

    The pipeline's build engine. Because project users do not directly interact with Jenkins, it's managed and locked.

    note

    There is no option to use existing Jenkins deployments as the pipeline engine.

  • Docker Registry:

    Out-of-the-box, the default target for your build-publish step is an internal Docker Registry. However, you can make configurations to push to a remote registry instead. The internal Docker Registry is only accessible from cluster nodes and cannot be directly accessed by users. Images are not persisted beyond the lifetime of the pipeline and should only be used in pipeline runs. If you need to access your images outside of pipeline runs, please push to an external registry.

  • Minio:

    Minio storage is used to store the logs for pipeline executions.

    note

    The managed Jenkins instance works statelessly, so don't worry about its data persistency. The Docker Registry and Minio instances use ephemeral volumes by default, which is fine for most use cases. If you want to make sure pipeline logs can survive node failures, you can configure persistent volumes for them, as described in data persistency for pipeline components.

Role-based Access Control for Pipelines​

If you can access a project, you can enable repositories to start building pipelines.

Only administrators, cluster owners or members, or project owners can configure version control providers and manage global pipeline execution settings.

Project members can only configure repositories and pipelines.

Setting up Pipelines​

Prerequisite​

Legacy Feature Flag:

Because the pipelines app was deprecated in favor of Fleet, you will need to turn on the feature flag for legacy features before using pipelines. Note that pipelines in Kubernetes 1.21+ are no longer supported.

  1. In the upper left corner, click ☰ > Global Settings.
  2. Click Feature Flags.
  3. Go to the legacy feature flag and click ⋮ > Activate.
  1. Configure version control providers
  2. Configure repositories
  3. Configure the pipeline

1. Configure Version Control Providers​

Before you can start configuring a pipeline for your repository, you must configure and authorize a version control provider:

  • GitHub
  • GitLab
  • Bitbucket

Select your provider's tab below and follow the directions.

  1. In the upper left corner, click ☰ > Cluster Management.
  2. Go to the cluster where you want to configure pipelines and click Explore.
  3. In the dropdown menu in the top navigation bar, select the project where you want to configure pipelines.
  4. In the left navigation bar, click Legacy > Project > Pipelines.
  5. Click the Configuration tab.
  6. Follow the directions displayed to Setup a GitHub application. Rancher redirects you to GitHub to set up an OAuth App in GitHub.
  7. From GitHub, copy the Client ID and Client Secret. Paste them into Rancher.
  8. If you're using GitHub for enterprise, select Use a private github enterprise installation. Enter the host address of your GitHub installation.
  9. Click Authenticate.

Result: After the version control provider is authenticated, you will be automatically re-directed to start configuring which repositories you want start using with a pipeline.

2. Configure Repositories​

After the version control provider is authorized, you are automatically re-directed to start configuring which repositories that you want start using pipelines with. Even if someone else has set up the version control provider, you will see their repositories and can build a pipeline.

  1. In the upper left corner, click ☰ > Cluster Management.

  2. Go to the cluster where you want to configure pipelines and click Explore.

  3. In the dropdown menu in the top navigation bar, select the project where you want to configure pipelines.

  4. In the left navigation bar, click Legacy > Project > Pipelines.

  5. Click on Configure Repositories.

  6. A list of repositories are displayed. If you are configuring repositories the first time, click on Authorize & Fetch Your Own Repositories to fetch your repository list.

  7. For each repository that you want to set up a pipeline, click on Enable.

  8. When you're done enabling all your repositories, click on Done.

Results: You have a list of repositories that you can start configuring pipelines for.

3. Configure the Pipeline​

Now that repositories are added to your project, you can start configuring the pipeline by adding automated stages and steps. For your convenience, there are multiple built-in step types for dedicated tasks.

  1. In the upper left corner, click ☰ > Cluster Management.

  2. Go to the cluster where you want to configure pipelines and click Explore.

  3. In the dropdown menu in the top navigation bar, select the project where you want to configure pipelines.

  4. In the left navigation bar, click Legacy > Project > Pipelines.

  5. Find the repository that you want to set up a pipeline for.

  6. Configure the pipeline through the UI or using a yaml file in the repository, i.e. .rancher-pipeline.yml or .rancher-pipeline.yaml. Pipeline configuration is split into stages and steps. Stages must fully complete before moving onto the next stage, but steps in a stage run concurrently. For each stage, you can add different step types. Note: As you build out each step, there are different advanced options based on the step type. Advanced options include trigger rules, environment variables, and secrets. For more information on configuring the pipeline through the UI or the YAML file, refer to the pipeline configuration reference.

    • If you are going to use the UI, select the vertical ⋮ > Edit Config to configure the pipeline using the UI. After the pipeline is configured, you must view the YAML file and push it to the repository.
    • If you are going to use the YAML file, select the vertical ⋮ > View/Edit YAML to configure the pipeline. If you choose to use a YAML file, you need to push it to the repository after any changes in order for it to be updated in the repository. When editing the pipeline configuration, it takes a few moments for Rancher to check for an existing pipeline configuration.

  7. Select which branch to use from the list of branches.

  8. Optional: Set up notifications.

  9. Set up the trigger rules for the pipeline.

  10. Enter a Timeout for the pipeline.

  11. When all the stages and steps are configured, click Done.

Results: Your pipeline is now configured and ready to be run.

Pipeline Configuration Reference​

Refer to this page for details on how to configure a pipeline to:

  • Run a script
  • Build and publish images
  • Publish catalog templates
  • Deploy YAML
  • Deploy a catalog app

The configuration reference also covers how to configure:

  • Notifications
  • Timeouts
  • The rules that trigger a pipeline
  • Environment variables
  • Secrets

Running your Pipelines​

Run your pipeline for the first time. Find your pipeline and select the vertical ⋮ > Run.

During this initial run, your pipeline is tested, and the following pipeline components are deployed to your project as workloads in a new namespace dedicated to the pipeline:

  • docker-registry
  • jenkins
  • minio

This process takes several minutes. When it completes, you can view each pipeline component from the project Workloads tab.

Triggering a Pipeline​

When a repository is enabled, a webhook is automatically set in the version control provider. By default, the pipeline is triggered by a push event to a repository, but you can modify the event(s) that trigger running the pipeline.

Available Events:

  • Push: Whenever a commit is pushed to the branch in the repository, the pipeline is triggered.
  • Pull Request: Whenever a pull request is made to the repository, the pipeline is triggered.
  • Tag: When a tag is created in the repository, the pipeline is triggered.
note

This option doesn't exist for Rancher's example repositories.

Modifying the Event Triggers for the Repository​

  1. In the upper left corner, click ☰ > Cluster Management.
  2. Go to the cluster where you want to configure pipelines and click Explore.
  3. In the dropdown menu in the top navigation bar, select the project where you want to configure pipelines.
  4. In the left navigation bar, click Legacy > Project > Pipelines.
  5. Find the repository where you want to modify the event triggers. Select the vertical ⋮ > Setting.
  6. Select which event triggers (Push, Pull Request or Tag) you want for the repository.
  7. Click Save.