Azure, Azure Pipelines, IaC

Using Containers to Share Terraform Modules and Deploy with Azure Pipelines

I’ve been using a container for running Terraform for a while but just for local development. More recently though the need to share modules has become more prevalent.

One solution for this is to use a container to not only share modules for development but for deployment as well. This also allows the containers to be versioned, limiting breaking changes affecting multiple pipelines at once.

In this post I am going to cover:

  • Building a container with shared terraform modules
  • Pushing the built container to Azure Container Registry
  • Configuring the dev environment to use the built container
  • Deploy infrastructure using the built container

NOTE: All of the code used here can be found on my GitHub including the shared modules.

Prerequisites

For this post I will running on Windows and using the following programs:

Building the Container

The container needs to not only have what is needed for development but what is needed to run as a container job in Azure Pipelines e.g. Node. The Microsoft Docs provide more detail about this.

The container is an Alpine Linux base with Node, PowerShell Core, Azure CLI and Terraform installed.

Dockerfile

ARG IMAGE_REPO=alpine
ARG IMAGE_VERSION=3
ARG TERRAFORM_VERSION
ARG POWERSHELL_VERSION
ARG NODE_VERSION=lts-alpine3.14

FROM node:${NODE_VERSION} AS node_base
RUN echo "NODE Version:" && node --version
RUN echo "NPM Version:" && npm --version

FROM ${IMAGE_REPO}:${IMAGE_VERSION} AS installer-env
ARG TERRAFORM_VERSION
ARG POWERSHELL_VERSION
ARG POWERSHELL_PACKAGE=powershell-${POWERSHELL_VERSION}-linux-alpine-x64.tar.gz
ARG POWERSHELL_DOWNLOAD_PACKAGE=powershell.tar.gz
ARG POWERSHELL_URL=https://github.com/PowerShell/PowerShell/releases/download/v${POWERSHELL_VERSION}/${POWERSHELL_PACKAGE}
RUN apk upgrade --update && \
    apk add --no-cache bash wget curl

# Terraform
RUN wget --quiet https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip && \
    unzip terraform_${TERRAFORM_VERSION}_linux_amd64.zip && \
    mv terraform /usr/bin
    
# PowerShell Core
RUN curl -s -L ${POWERSHELL_URL} -o /tmp/${POWERSHELL_DOWNLOAD_PACKAGE}&& \
    mkdir -p /opt/microsoft/powershell/7 && \
    tar zxf /tmp/${POWERSHELL_DOWNLOAD_PACKAGE} -C /opt/microsoft/powershell/7 && \
    chmod +x /opt/microsoft/powershell/7/pwsh 

FROM ${IMAGE_REPO}:${IMAGE_VERSION} 
ENV NODE_HOME /usr/local/bin/node
# Copy only the files we need from the previous stages
COPY --from=installer-env ["/usr/bin/terraform", "/usr/bin/terraform"]
COPY --from=installer-env ["/opt/microsoft/powershell/7", "/opt/microsoft/powershell/7"]
RUN ln -s /opt/microsoft/powershell/7/pwsh /usr/bin/pwsh
COPY --from=node_base ["${NODE_HOME}", "${NODE_HOME}"]

# Copy over Modules
RUN mkdir modules
COPY modules modules

LABEL maintainer="Coding With Taz"
LABEL "com.azure.dev.pipelines.agent.handler.node.path"="${NODE_HOME}"

ENV APK_DEV "gcc libffi-dev musl-dev openssl-dev python3-dev make"
ENV APK_ADD "bash sudo shadow curl py3-pip graphviz git"
ENV APK_POWERSHELL="ca-certificates less ncurses-terminfo-base krb5-libs libgcc libintl libssl1.1 libstdc++ tzdata userspace-rcu zlib icu-libs"
# Install additional packages
RUN apk upgrade --update && \
    apk add --no-cache --virtual .pipeline-deps readline linux-pam && \
    apk add --no-cache --virtual .build ${APK_DEV} && \
    apk add --no-cache ${APK_ADD} ${APK_POWERSHELL} && \
    # Install Azure CLI
    pip --no-cache-dir install --upgrade pip && \
    pip --no-cache-dir install wheel && \
    pip --no-cache-dir install azure-cli && \
    apk del .build && \
    apk del .pipeline-deps 

RUN echo "PS1='\n\[\033[01;35m\][\[\033[0m\]Terraform\[\033[01;35m\]]\[\033[0m\]\n\[\033[01;35m\][\[\033[0m\]\[\033[01;32m\]\w\[\033[0m\]\[\033[01;35m\]]\[\033[0m\]\n \[\033[01;33m\]->\[\033[0m\] '" >> ~/.bashrc 
CMD tail -f /dev/null

The container can be built locally using the docker build command and providing the PowerShell and Terraform versions e.g.

docker build --build-arg TERRAFORM_VERSION="1.0.10" --build-arg POWERSHELL_VERSION="7.1.5" -t my-terraform .

Pushing the container to Azure Container Registry

Next thing to do is to build the container and push it to the Azure Container Registry (if you need to know how to set that up in Azure DevOps see my previous post on Configuring ACR). In this pipeline I have also added a Snyk scan to check for vulnerabilities in my container (happy to report there wasn’t any at the time of writing). If you are not familiar with Snyk I recommend you check out their website.

For the build number I have used the version of Terraform and then the date and revision but you can use whatever makes sense for example you could use Semver.

I also setup some pipeline variables for the container registry connection and the container registry name e.g. <your registry>.azurecr.io

trigger: 
    - main

pr: none

name: $(terraformVersion)_$(Date:yyyyMMdd)$(Rev:.r)

variables:
 dockerFilePath: dockerfile
 imageRepository: iac/terraform
 terraformVersion: 1.0.10
 powershellVersion: 7.1.5

pool:
  vmImage: "ubuntu-latest"

steps:
  - task: Docker@2
    displayName: "Build Terraform Image"
    inputs:
      containerRegistry: '$(containerRegistryConnection)'
      repository: '$(imageRepository)'
      command: 'build'
      Dockerfile: '$(dockerfilePath)'
      arguments: '--build-arg TERRAFORM_VERSION="$(terraformVersion)" --build-arg POWERSHELL_VERSION="$(powershellVersion)"'
      tags: | 
        $(Build.BuildNumber)
  - task: SnykSecurityScan@1
    inputs:
      serviceConnectionEndpoint: 'Snyk'
      testType: 'container'
      dockerImageName: '$(containerRegistry)/$(imageRepository):$(Build.BuildNumber)'
      dockerfilePath: '$(dockerfilePath)'
      monitorWhen: 'always'
      severityThreshold: 'high'
      failOnIssues: true
  - task: Docker@2
    displayName: "Build and Push Terraform Image"
    inputs:
      containerRegistry: '$(containerRegistryConnection)'
      repository: '$(imageRepository)'
      command: 'Push'
      Dockerfile: '$(dockerfilePath)'
      tags: | 
        $(Build.BuildNumber)

Once the container is built it can be viewed in the Azure Portal inside your Azure Container Registry.

Configuring the Dev Environment

Now the container has been created and pushed to the Azure Container Registry the next job is to configure Visual Studio Code.

To start with we need to make sure the extension Remote Containers is installed in Visual Studio Code

In the project where you want to use the container, create a folder called .devcontainer and then a file inside the folder called devcontainer.json and add the following (updating the container registry and container details e.g. name, version, etc.)

// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
// https://github.com/microsoft/vscode-dev-containers/tree/v0.205.1/containers/docker-existing-dockerfile
{
	"name": "Terraform Dev",

	// Sets the run context to one level up instead of the .devcontainer folder.
	"context": "..",

	// Update the 'dockerFile' property if you aren't using the standard 'Dockerfile' filename.
	"image": "<your container registry>.azurecr.io/iac/terraform:1.0.10_20211108.1",

	// Set *default* container specific settings.json values on container create.
	"settings": {},
	
	// Add the IDs of extensions you want installed when the container is created.
	"extensions": [
		"ms-vscode.azure-account",
		"ms-azuretools.vscode-azureterraform",
		"hashicorp.terraform",
		"ms-azure-devops.azure-pipelines"
	]
}

NOTE: You may notice that there is a number of extensions in the above config. I use these extensions in Visual Studio Code for Terraform, Azure Pipelines, etc. and therefore they would also need installing in order to make use of them in the container environment.

TIP: If you right-click on an extension in Visual Studio Code and select ‘Copy Extension ID’ you can easily get the extension information you need to add other extensions to the list.

Now, make sure to login to the Azure Container Registry (either in another window or the terminal in Visual Studio Code) with the Azure CLI for authentication e.g.

az acr login -n <your container registry name>

This needs to be done to be able to pull down the container. Once the login is successful, select the icon in the bottom left of Visual Studio Code to ‘Open a Remote Window’

Then select ‘Reopen in Container’ this will download the container from the Azure Container Registry and load up the project in the container (this can take a minute or so first time).

Once the project is loaded you can create Terraform files as normal and take advantage of the shared modules inside the container.

So lets create a small example. I work a lot in Azure so I am using a shared module to create an Azure Function and another module to format the naming convention for the resources.

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 2.83"
    }
  }
  backend "local" {}
  required_version = ">= 1.0.10"
}

provider "azurerm" {
  features {}
}


module "rgname" {
    source        = "/modules/naming"
    name          = "myapp"
    env           = "rg-${var.env}"
    resource_type = ""
    location      = var.location
    separator     = "-"
}

resource "azurerm_resource_group" "rg" {
  name     = module.rgname.result
  location = "uksouth"
}

module "funcApp" {
  source                    = "/modules/linux_azure_function"
  resource_group            = azurerm_resource_group.rg.name
  resource_group_location   = azurerm_resource_group.rg.location
  env                       = var.env
  appName                   = var.appName
  funcWorkerRuntime         = "dotnet-isolated"
  dotnetVersion             = "v5.0"
  additionalFuncAppSettings = {
    mysetting = "somevalue"
  }
  tags                      = var.tags
}

From the terminal window I can now authenticate to Azure by logging in via the CLI

az login

Then I can run the terraform commands

terraform init
terraform plan

This produces the terraform plan for the resources that would be created.

Deploy Infrastructure Using the Container

So now I have created a new terraform configuration its time to deploy the changes using the same container.

To do this I am using Azure Pipelines YAML. There are several parts to the pipeline, firstly, in order to store the state for the pipeline there needs to be an Azure Storage Account to store the state file. I like to add this to the pipeline using Azure CLI so that the account is created if it doesn’t exist but also updates it if there are changes.

 - task: AzureCLI@2
    displayName: 'Create/Update State File Storage'
    inputs:
        azureSubscription: '$(subscription)'
        scriptType: bash
        scriptLocation: inlineScript
        inlineScript: |
          az group create --location $(location) --name $(terraformGroup)
          az storage account create --name $(terraformStorageName) --resource-group $(terraformGroup) --location $(location) --sku $(terraformStorageSku) --min-tls-version TLS1_2 --https-only true --allow-blob-public-access false
          az storage container create --name $(terraformContainerName) --account-name $(terraformStorageName)
        addSpnToEnvironment: false

The terraform backend configuration is set to local for development and so I need a step in the pipeline to update it to use backend “azurerm”.

  - bash: |
      sed -i 's/backend "local" {}/backend "azurerm" {}/g' main.tf
    displayName: 'Update Backend in terraform file'

For the Terraform commands I tend to use the Microsoft Terraform Tasks with additional command options for the plan file

- task: TerraformTaskV2@2
    displayName: 'Terraform Init'
    inputs:
      backendServiceArm: '$(subscription)'
      backendAzureRmResourceGroupName: '$(terraformGroup)'
      backendAzureRmStorageAccountName: '$(terraformStorageName)'
      backendAzureRmContainerName: '$(terraformContainerName)'
      backendAzureRmKey: '$(terraformStateFilename)'
  - task: TerraformTaskV2@2
    displayName: 'Terraform Plan'
    inputs:
      command: plan
      commandOptions: '-out=tfplan'
      environmentServiceNameAzureRM: '$(subscription)'
  - task: TerraformTaskV2@2
    displayName: 'Terraform Apply'
    inputs:
      command: apply
      commandOptions: '-auto-approve tfplan'
      environmentServiceNameAzureRM: '$(subscription)'

So, putting it all together the whole pipeline looks likes this:

trigger:
   - main
 
pr: none
parameters:
  - name: env
    displayName: 'Environment'
    type: string
    default: 'dev'
    values:
      - dev
      - test
      - prod
  - name: location
    displayName: 'Resource Location'
    type: string
    default: 'uksouth'
  - name: appName
    displayName: 'Application Name'
    type: string
    default: 'myapp'
  - name: tags 
    displayName: 'Tags'
    type: object 
    default: 
     Environment: "dev"
     Project: "Demo"
variables:
  isMain: $[eq(variables['Build.SourceBranch'], 'refs/heads/main')]
  location: 'uksouth'
  terraformGroup: 'rg-dev-terraform-uksouth'
  terraformStorageName: 'devterraformuksouth2329'
  terraformStorageSku: 'Standard_LRS'
  terraformContainerName: 'infrastructure'
  terraformStateFilename: 'deploy.tfstate'
  
jobs:
- job: infrastructure
  displayName: 'Build Infrastructure'
  pool:
    vmImage: ubuntu-latest
  container:
    image: $(containerRegistry)/iac/terraform:1.0.10_20211108.1
    endpoint: 'ACR Connection'
  steps:
  - task: AzureCLI@2
    displayName: 'Create/Update State File Storage'
    inputs:
        azureSubscription: '$(subscription)'
        scriptType: bash
        scriptLocation: inlineScript
        inlineScript: |
          az group create --location $(location) --name $(terraformGroup)
          az storage account create --name $(terraformStorageName) --resource-group $(terraformGroup) --location $(location) --sku $(terraformStorageSku) --min-tls-version TLS1_2 --https-only true --allow-blob-public-access false
          az storage container create --name $(terraformContainerName) --account-name $(terraformStorageName)
        addSpnToEnvironment: false
  - bash: |
      sed -i 's/backend "local" {}/backend "azurerm" {}/g' main.tf
    displayName: 'Update Backend in terraform file'
  - template: 'autovars.yml'
    parameters:
      env: ${{ parameters.env }}
      location: ${{ parameters.location }}
      appName: ${{ parameters.appName }}
      tags: ${{ parameters.tags }}
  - task: TerraformTaskV2@2
    displayName: 'Terraform Init'
    inputs:
      backendServiceArm: '$(subscription)'
      backendAzureRmResourceGroupName: '$(terraformGroup)'
      backendAzureRmStorageAccountName: '$(terraformStorageName)'
      backendAzureRmContainerName: '$(terraformContainerName)'
      backendAzureRmKey: '$(terraformStateFilename)'
  - task: TerraformTaskV2@2
    displayName: 'Terraform Plan'
    inputs:
      command: plan
      commandOptions: '-out=tfplan'
      environmentServiceNameAzureRM: '$(subscription)'
  - task: TerraformTaskV2@2
    displayName: 'Terraform Apply'
    inputs:
      command: apply
      commandOptions: '-auto-approve tfplan'
      environmentServiceNameAzureRM: '$(subscription)'

As with the container build pipeline I used some pipeline variables here for the subscription connection and the container registry e.g. <your registry>.azurecr.io

After the pipeline ran, a quick check in the Azure Portal shows the resources were created as expected

Final Thoughts

I really like using containers for local development and with the remote containers extension for Visual Studio Code its great to be able to run from within a container and share code in this way. I am sure that other things could be shared using this method too.

Being able to version the containers and isolate breaking changes across multiple pipelines is also a bonus. I expect this process could be better, maybe even include pinning of provider versions in Terraform, etc. but its a good start.

Azure, Azure Pipelines

Azure Pipelines – Multistage YAML

Azure Pipelines YAML allows us to create PaC (Pipeline as Code) to build and deploy applications to multiple stages e.g. Staging, Production.

To demonstrate this process I will cover the following:

  • Build a simple web application with UI tests
  • Publish the web application to an ACR (Azure Container Registry)
  • Create an Azure Web App with IaC (Infrastructure as Code)
  • Deploy the web application container to the Azure Web App
  • Run basic UI tests on multiple stages

This article assumes that you are familiar with building YAML pipelines in Azure DevOps Pipelines.

The Web Application

For simplicity I have used the default ASP.NET Core Web Application in Visual Studio 2019 with Docker Support enabled for Linux to create the web application.

The only thing added to the default web application is a few UI tests using Selenium. You can find all the code used and the deployment files on my GitHub.

The Pipeline

After creating a new pipeline in Azure Pipelines, I need to configure the Azure and ACR connection variables in the pipeline UI.

If you need to know how to configure the ACR service connection see my previous article Configure ACR – Azure DevOps.

Build Image

Now everything is configured, I can create the initial YAML to build and push the application to an ACR.

As this will be a multistage pipeline I will create the first Stage to build and push the image.

trigger:
- master

resources:
- repo: self

variables:  
  imageRepository: 'multistagepipelines'   
  tag: '$(Build.BuildId)' 
  vmImageName: 'ubuntu-latest'
  uiTestFolder: 'uitests'

stages:
- stage: Build
  displayName: Build and push stage
  jobs:  
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)  
    steps:
      - task: Docker@2
        displayName: Build and push an image to container registry
        inputs:
          containerRegistry: 'ACR Connection'
          repository: '$(imageRepository)'
          command: 'buildAndPush'
          Dockerfile: '**/Dockerfile'
          tags: |
            latest
            $(tag)

Now I can run this pipeline and see if it was successful.

And I can check the ACR in Azure to confirm the image has successfully been created.

Define the Web App

Now I have the image uploaded to the ACR, I need to define the Azure Web App that I will be deploying to.

For this I will use an ARM (Azure Resource Manager) template.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "siteName": {
            "type": "string",
            "metadata": {
                "description": "The unique name of your Web Site."
            }
        },
        "appImageName": {
            "type": "string",
            "metadata": {
                "description": "The name of the container image for this web app"
            }
        },
        "containerRegistryName": {
            "type": "string",
            "metadata": {
                "description": "The name of the azure container registry that contains the webapp"
            }
        },
        "containerRegistryUserName": {
            "type": "string",
            "metadata": {
                "description": "The user name to access the azure container registry"
            }
        },
        "containerRegistryPassword": {
            "type": "string",
            "metadata": {
                "description": "The password to access the azure container registry"
            }
        }

    },
    "variables": {
        "hostingPlanName": "[concat('hpn-',  parameters('siteName'))]",
        "siteApiVersion": "2019-08-01"
    },
    "resources": [
        {
            "type": "Microsoft.Web/serverfarms",
            "apiVersion": "[variables('siteApiVersion')]",
            "name": "[variables('hostingPlanName')]",
            "location": "[resourceGroup().location]",
            "properties": {
                "name": "[variables('hostingPlanName')]",
                "workerSizeId": "1",
                "reserved": true,
                "numberOfWorkers": "1"
            },
            "sku": {
                "Tier": "Standard",
                "Name": "S1"
            },
            "kind": "linux"
        },
        {
            "name": "[parameters('siteName')]",
            "type": "Microsoft.Web/sites",
            "apiVersion": "[variables('siteApiVersion')]",
            "kind": "app,linux,container",
            "location": "[resourceGroup().location]",
            "tags": {
                "hostingPlan": "[variables('hostingPlanName')]",
                "displayName": "[parameters('siteName')]"
            },
            "dependsOn": [
                "[variables('hostingPlanName')]"
            ],
            "properties": {
                "name": "[parameters('siteName')]",
                "serverFarmId": "[variables('hostingPlanName')]",
                "siteConfig": {
                    "use32BitWorkerProcess": false,
                    "http20Enabled": true,
                    "minTlsVersion": "1.2",
                    "alwaysOn": true,
                    "linuxFxVersion": "[concat('DOCKER|', parameters('appImageName'))]",
                    "appSettings": [
                        {
                            "name": "DOCKER_REGISTRY_SERVER_USERNAME",
                            "value": "[parameters('containerRegistryUserName')]"
                        },
                        {
                            "name": "DOCKER_REGISTRY_SERVER_URL",
                            "value": "[concat('https://',parameters('containerRegistryName'))]"
                        },
                        {
                            "name": "DOCKER_REGISTRY_SERVER_PASSWORD",
                            "value": "[parameters('containerRegistryPassword')]"
                        }
                    ]
                }
            }
        }
    ],
    "outputs": {}
}

There are few things to note in this template, firstly that we are deploying to a linux container so the website configuration is a little different to normal. The kind property needs to include more information than just app.

"kind": "app,linux,container"

And the reserved property must be set to true.

 "reserved": true

There are also a couple of settings that aren’t really documented in the Microsoft Docs to configure the app settings to connect to the ACR to retrieve the image. Adding these appSettings will setup the connection.

"appSettings": [
 {
   "name": "DOCKER_REGISTRY_SERVER_USERNAME",
   "value": "[parameters('containerRegistryUserName')]"
 },
 {
   "name": "DOCKER_REGISTRY_SERVER_URL",
   "value": "[concat('https://',parameters('containerRegistryName'))]"
 },
 {
   "name": "DOCKER_REGISTRY_SERVER_PASSWORD",
   "value": "[parameters('containerRegistryPassword')]"
  }
]

Publish Template

The first thing to change in the pipeline is to add a step to upload the ARM template to an artifact to use later in the deployment.

Adding a PublishBuildArtifacts task to the build steps will perform the artifact creation.

- task: PublishBuildArtifacts@1
  displayName: Publish ARM template
  inputs:
    PathtoPublish: 'deploy.json'
    ArtifactName: 'template'
    publishLocation: 'Container'

Publish Tests

You may have noticed in the pipeline that I used “Jobs” and created a single job, this could be seen as unnecessary, but now I am going to add another job that will run in parallel with the Build Job.

So I need to add some tasks to build my UI tests. I’ve also added a variable “vmWindowsImageName” as for this job I am going to use a windows image. The test project is .NET Core 3.1 so I will use the DotNetCoreCLI tasks to restore packages and build the tests.

- job: BuildTests
  displayName: Build UI Tests
  pool:
    vmImage: $(vmWindowsImageName)
  steps:
    - task: DotNetCoreCLI@2
      displayName: Restore Packages
      inputs:
        command: 'restore'
        projects: 'multistagepipelinestests/*.csproj'
    - task: DotNetCoreCLI@2
      displayName: Build Tests
      inputs:
        command: 'build'
        projects: '**/multistagepipelinestests.csproj'
        arguments: '--configuration Release -o $(Build.ArtifactStagingDirectory)/uitests'

As with the ARM template, the UI tests need publishing to use later.

- task: PublishBuildArtifacts@1
  displayName: Publish UI Tests
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)/$(uiTestFolder)'
    ArtifactName: $(uiTestFolder)
    publishLocation: 'Container'

Deployment

Now the pipeline builds and publishes the necessary artifacts to the pipeline and the ACR, I can now add a new stage to deploy the application.

This new stage uses a special job, a ‘deployment’ job and uses a strategy. The Microsoft Docs have a lot of information about different strategies, for this I will use the ‘runonce’ strategy as the other strategies are not supported here.

- stage: Staging
  displayName: Deploy to Staging
  jobs:
  - deployment: DeployWeb
    displayName: Deploy Web App
    pool:
     vmImage: $(vmWindowsImageName)
    environment: Staging
    variables:
      siteName: staging-taz-app
      siteResourceGroup: stag-taz-webapp
      siteLocation: UK South
      appImageName: $(containerRegistryName)/$(imageRepository):latest
      baseSiteUrl: 'https://$(siteName).azurewebsites.net/'
    strategy:
      runOnce:       
        deploy:
          steps:

With the job and strategy configured, I can now add the first step to execute the ARM template and create the Web App.

- task: AzureResourceManagerTemplateDeployment@3
    displayName: Create or Update Azure Web App
    inputs:
      deploymentScope: 'Resource Group'
      azureResourceManagerConnection: $(SubscriptionName)
      subscriptionId: $(subscriptionId)
      action: 'Create Or Update Resource Group'
      resourceGroupName: $(siteResourceGroup)
      location: $(siteLocation)
      templateLocation: 'Linked artifact'
      csmFile: '$(Pipeline.Workspace)/template/deploy.json'
      overrideParameters: '-siteName $(siteName) -appImageName $(appImageName) -containerRegistryName $(containerRegistryName) -containerRegistryUserName $(containerRegistryUserName) -containerRegistryPassword $(containerRegistryPassword)'
      deploymentMode: 'Incremental'

Once the Web App is created I can deploy the application container into the new Web App. As this is a container application I will use the AzureWebAppContainer task.

- task: AzureWebAppContainer@1
  displayName: Deploy Application
  inputs:
    azureSubscription: $(SubscriptionName)
    appName: '$(siteName)'
    containers: '$(appImageName)'

Once the app is deployed I can then run the UI tests, but first I’ll need to add a FileTranform task to make sure my settings file has the correct URL configured to run the tests against.

- task: FileTransform@2
  displayName: Configure Staging
  inputs:
    folderPath: '$(Pipeline.Workspace)'
    xmlTransformationRules: ''
    jsonTargetFiles: '**/*settings.json'

If you want to check that the settings file correctly transformed you can add a simple PowerShell task to output the file contents.

- task: PowerShell@2
  inputs:
    targetType: 'inline'
    script: 'Get-Content -Path $(Pipeline.Workspace)/$(uiTestFolder)/testsettings.json'
    pwsh: true

And now a task to run the UI tests, for this I will use the VSTest task to run and publish the test results to the Azure Pipeline UI.

- task: VSTest@2
  displayName: Run UI Tests
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: |
      ***tests.dll
      !***TestAdapter.dll
      !**obj**
    searchFolder: '$(Pipeline.Workspace)/$(uiTestFolder)'
    uiTests: true
    testRunTitle: 'Basic UI Tests'

There have been a lot of changes added, so let’s see the full pipeline so far:

trigger:
- master

resources:
- repo: self

variables:  
  imageRepository: 'multistagepipelines'   
  tag: '$(Build.BuildId)' 
  vmImageName: 'ubuntu-latest'
  vmWindowsImageName: 'windows-latest'
  uiTestFolder: 'uitests'

stages:
- stage: Build
  displayName: Build and push stage
  jobs:  
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)
    steps:
      - task: Docker@2
        displayName: Build and push an image to container registry
        inputs:
          containerRegistry: 'ACR Connection'
          repository: '$(imageRepository)'
          command: 'buildAndPush'
          Dockerfile: '**/Dockerfile'
          tags: |
            latest
            $(tag)
      - task: PublishBuildArtifacts@1
        displayName: Publish ARM template
        inputs:
          PathtoPublish: 'deploy.json'
          ArtifactName: 'template'
          publishLocation: 'Container'
  - job: BuildTests
    displayName: Build UI Tests
    pool:
      vmImage: $(vmWindowsImageName)
    steps:
      - task: DotNetCoreCLI@2
        displayName: Restore Packages
        inputs:
          command: 'restore'
          projects: 'multistagepipelinestests/*.csproj'
      - task: DotNetCoreCLI@2
        displayName: Build Tests
        inputs:
          command: 'build'
          projects: '**/multistagepipelinestests.csproj'
          arguments: '--configuration Release -o $(Build.ArtifactStagingDirectory)/uitests'
      - task: PublishBuildArtifacts@1
        displayName: Publish UI Tests
        inputs:
          PathtoPublish: '$(Build.ArtifactStagingDirectory)/$(uiTestFolder)'
          ArtifactName: $(uiTestFolder)
          publishLocation: 'Container'
- stage: Staging
  displayName: Deploy to Staging
  jobs:
  - deployment: DeployWeb
    displayName: Deploy Web App
    pool:
     vmImage: $(vmWindowsImageName)
    environment: Staging
    variables:
      siteName: staging-taz-app
      siteResourceGroup: stag-taz-webapp
      siteLocation: UK South
      appImageName: $(containerRegistryName)/$(imageRepository):latest
      baseSiteUrl: 'https://$(siteName).azurewebsites.net/'
    strategy:
      runOnce:       
        deploy:
          steps:
          - task: AzureResourceManagerTemplateDeployment@3
            displayName: Create or Update Azure Web App
            inputs:
              deploymentScope: 'Resource Group'
              azureResourceManagerConnection: $(SubscriptionName)
              subscriptionId: $(subscriptionId)
              action: 'Create Or Update Resource Group'
              resourceGroupName: $(siteResourceGroup)
              location: $(siteLocation)
              templateLocation: 'Linked artifact'
              csmFile: '$(Pipeline.Workspace)/template/deploy.json'
              overrideParameters: '-siteName $(siteName) -appImageName $(appImageName) -containerRegistryName $(containerRegistryName) -containerRegistryUserName $(containerRegistryUserName) -containerRegistryPassword $(containerRegistryPassword)'
              deploymentMode: 'Incremental'
          - task: AzureWebAppContainer@1
            displayName: Deploy Application
            inputs:
              azureSubscription: $(SubscriptionName)
              appName: '$(siteName)'
              containers: '$(appImageName)'
          - task: FileTransform@2
            displayName: Configure Staging
            inputs:
              folderPath: '$(Pipeline.Workspace)'
              xmlTransformationRules: ''
              jsonTargetFiles: '**/*settings.json'
          - task: VSTest@2
            displayName: Run UI Tests
            inputs:
              testSelector: 'testAssemblies'
              testAssemblyVer2: |
                ***tests.dll
                !***TestAdapter.dll
                !**obj**
              searchFolder: '$(Pipeline.Workspace)/$(uiTestFolder)'
              uiTests: true
              testRunTitle: 'Basic UI Tests'

Enhance the Pipeline

Currently the pipeline:

  • Builds a web application image and uploads it to an ACR
  • Deploys an Azure Web App using an ARM Template
  • Deploys the image into the Azure Web App
  • And runs UI tests against the newly deployed application

This is great but I would guess most of us don’t just have one environment that we need to deploy to and will need at least another one and maybe a manual intervention step too.

To create another environment I could just copy and paste the ‘Staging’ stage, rename it and update the variables. Whilst this approach would work, it would introduce a maintenance overhead we don’t want.

Fortunately Azure Pipelines YAML includes Templates for variables, jobs, steps and stages to handle this.

So, I will move the steps for the ‘Staging’ deployment into a template and call it web-deploy-steps.yml. The template file will look like:

steps:
- task: AzureResourceManagerTemplateDeployment@3
  displayName: Create or Update Azure Web App
  inputs:
    deploymentScope: 'Resource Group'
    azureResourceManagerConnection: $(SubscriptionName)
    subscriptionId: $(subscriptionId)
    action: 'Create Or Update Resource Group'
    resourceGroupName: $(siteResourceGroup)
    location: $(siteLocation)
    templateLocation: 'Linked artifact'
    csmFile: '$(Pipeline.Workspace)/template/deploy.json'
    overrideParameters: '-siteName $(siteName) -appImageName $(appImageName) -containerRegistryName $(containerRegistryName) -containerRegistryUserName $(containerRegistryUserName) -containerRegistryPassword $(containerRegistryPassword)'
    deploymentMode: 'Incremental'
- task: AzureWebAppContainer@1
  displayName: Deploy Application
  inputs:
    azureSubscription: $(SubscriptionName)
    appName: '$(siteName)'
    containers: '$(appImageName)'
- task: FileTransform@2
  displayName: Configure Staging
  inputs:
    folderPath: '$(Pipeline.Workspace)'
    xmlTransformationRules: ''
    jsonTargetFiles: '**/*settings.json'
- task: PowerShell@2
  inputs:
    targetType: 'inline'
    script: 'Get-Content -Path $(Pipeline.Workspace)/$(uiTestFolder)/testsettings.json'
    pwsh: true
- task: VSTest@2
  displayName: Run UI Tests
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: |
      **\*tests.dll
      !**\*TestAdapter.dll
      !**\obj\**
    searchFolder: '$(Pipeline.Workspace)/$(uiTestFolder)'
    uiTests: true
    testRunTitle: 'Basic UI Tests'

Now I can update the ‘Staging’ stage to use the new template.

- stage: Staging
  displayName: Deploy to Staging
  jobs:
  - deployment: DeployWeb
    displayName: Deploy Web App
    pool:
     vmImage: $(vmWindowsImageName)
    environment: Staging
    variables:
      siteName: staging-taz-app
      siteResourceGroup: stag-taz-webapp
      siteLocation: UK South
      appImageName: $(containerRegistryName)/$(imageRepository):latest
      baseSiteUrl: 'https://$(siteName).azurewebsites.net/'
    strategy:
      runOnce:       
        deploy:
          steps:
          - template: web-deploy-steps.yml

It is now easy to add another stage using the same steps. I’ll add a production stage and update the variables.

- stage: Production
  displayName: Deploy to Production
  jobs:
  - deployment: DeployWeb
    displayName: Deploy Web App
    pool:
     vmImage: $(vmWindowsImageName)
    environment: Production
    variables:
      siteName: production-taz-app
      siteResourceGroup: prod-taz-webapp
      siteLocation: UK South
      appImageName: $(containerRegistryName)/$(imageRepository):latest
      baseSiteUrl: 'https://$(siteName).azurewebsites.net/'
    strategy:
      runOnce:       
        deploy:
          steps:
          - template: web-deploy-steps.yml

The full pipeline with the template now looks like:

trigger:
- master

resources:
- repo: self

variables:  
  imageRepository: 'multistagepipelines'   
  tag: '$(Build.BuildId)' 
  vmImageName: 'ubuntu-latest'
  vmWindowsImageName: 'windows-latest'
  uiTestFolder: 'uitests'

stages:
- stage: Build
  displayName: Build and push stage
  jobs:  
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)  
    steps:
      - task: Docker@2
        displayName: Build and push an image to container registry
        inputs:
          containerRegistry: 'ACR Connection'
          repository: '$(imageRepository)'
          command: 'buildAndPush'
          Dockerfile: '**/Dockerfile'
          tags: |
            latest
            $(tag)
      - task: PublishBuildArtifacts@1
        displayName: Publish ARM template
        inputs:
          PathtoPublish: 'deploy.json'
          ArtifactName: 'template'
          publishLocation: 'Container'
  - job: BuildTests
    displayName: Build UI Tests
    pool:
      vmImage: $(vmWindowsImageName)
    steps:
      - task: DotNetCoreCLI@2
        displayName: Restore Packages
        inputs:
          command: 'restore'
          projects: 'multistagepipelinestests/*.csproj'
      - task: DotNetCoreCLI@2
        displayName: Build Tests
        inputs:
          command: 'build'
          projects: '**/multistagepipelinestests.csproj'
          arguments: '--configuration Release -o $(Build.ArtifactStagingDirectory)/uitests'
      - task: PublishBuildArtifacts@1
        displayName: Publish UI Tests
        inputs:
          PathtoPublish: '$(Build.ArtifactStagingDirectory)/$(uiTestFolder)'
          ArtifactName: $(uiTestFolder)
          publishLocation: 'Container'
- stage: Staging
  displayName: Deploy to Staging
  jobs:
  - deployment: DeployWeb
    displayName: Deploy Web App
    pool:
     vmImage: $(vmWindowsImageName)
    environment: Staging
    variables:
      siteName: staging-taz-app
      siteResourceGroup: stag-taz-webapp
      siteLocation: UK South
      appImageName: $(containerRegistryName)/$(imageRepository):latest
      baseSiteUrl: 'https://$(siteName).azurewebsites.net/'
    strategy:
      runOnce:       
        deploy:
          steps:
          - template: web-deploy-steps.yml
- stage: Production
  displayName: Deploy to Production
  jobs:
  - deployment: DeployWeb
    displayName: Deploy Web App
    pool:
     vmImage: $(vmWindowsImageName)
    environment: Production
    variables:
      siteName: production-taz-app
      siteResourceGroup: prod-taz-webapp
      siteLocation: UK South
      appImageName: $(containerRegistryName)/$(imageRepository):latest
      baseSiteUrl: 'https://$(siteName).azurewebsites.net/'
    strategy:
      runOnce:       
        deploy:
          steps:
          - template: web-deploy-steps.yml

Review Output

Now the pipeline has ran, let’s check the results.

And let’s see if the resources were deployed into Azure.

Approvals and Checks

If the stage needs a manual intervention or approval step you can configure them in Azure Pipelines, just select ‘Environments’.

Once the list of environments is displayed you can select the one you need to add approvals and checks to e.g. Production.

Selecting the 3 dots on the right hand side and then selecting ‘Approvals and checks’ will allow a variety of options to be added.

There are a number of checks that can be added, here I will just select approvals.

Approvals simply need the users or groups that can approve the stage you want to control.

There are a few more settings for approvals, how many need to approve, approval timeout, etc. but I am not going to go into detail about them.

Conclusion

Azure Pipelines YAML provides a flexible way to create build and deployment pipelines that can be source controlled. Changes can be approved, tracked and are visible to everyone instead of a change via a UI that goes unnoticed and difficult to track if there is a problem caused by a change.

Being able to control the full application deployment flow this way is very powerful and allows the whole team to understand how their application is built and deployed.