Tag: Azure DevOps

Azure Pipelines, Bicep, DevOps, IaC

Passing Parameters to Bicep from Azure Pipelines

Posted on by Mark Johnson

In previous posts I’ve used various techniques to supply parameters to Bicep from Azure Pipelines YAML and overriding values in a parameters JSON file so I thought I would collate them in one post.

Technique #1

Using multiple template files to handle complex types and standard types separately and create a variable for each parameter that can then be used for File Transform of a parameters JSON file.

parameters:
- name: tags
  displayName: 'Tags'
  type: object
  default:
     Environment: "prod"
     Resource: "AKS"
     Project: "Demo"
 - name: clusterName
    displayName: 'Name of the AKS Cluster'
    type: string
    default: 'demo'
  - name: nodeVmSize
    displayName: 'VM Size for the Nodes'
    type: string
    default: 'Standard_D2s_V3'
    values:
      - 'Standard_D2s_V3'
      - 'Standard_DS2_v2'
      - 'Standard_D4s_V3'
      - 'Standard_DS3_v2'
      - 'Standard_DS4_v2'
      - 'Standard_D8s_v3'
  - name: nodeCount
    displayName: 'The number of nodes'
    type: number
    default: 3

- template: objectparameters.yml
  parameters:
    tags: ${{ parameters.tags }}
- template: parameters.yml
  parameters:
    clusterName: ${{ parameters.clusterName }}
    nodeVmSize: ${{ parameters.nodeVmSize }}
    nodeCount: ${{ parameters.nodeCount }}
- task: FileTransform@2
  displayName: "Transform Parameters"
  inputs:
    folderPath: '$(System.DefaultWorkingDirectory)'
    xmlTransformationRules: ''
    jsonTargetFiles: 'deploy.parameters.json'

objectParameters.yml

parameters:
  - name: tags
    type: object 
steps:
- ${{ each item in parameters }}:
  - bash: |
      value='${{ convertToJson(item.value) }}'
      echo '##vso[task.setvariable variable=parameters.${{ item.key }}.value]'$value
    displayName: "Create Variable ${{ item.key }}"

parameters.yml

parameters:
  - name: clusterName
    type: string
  - name: nodeVmSize
    type: string
  - name: nodeCount
    type: number
 
steps:
- ${{ each item in parameters }}: 
    - bash: |
        echo '##vso[task.setvariable variable=parameters.${{ item.key }}.value]${{ item.value }}'
      displayName: "Create Variable ${{ item.key }}"

Technique #2

Using a PowerShell step to read all parameters and create a variable for each parameter that can then be used for File Transform of a parameters JSON file.

parameters:
- name: tags
  displayName: 'Tags'
  type: object
  default:
     Environment: "prod"
     Resource: "AKS"
     Project: "Demo"
 - name: clusterName
    displayName: 'Name of the AKS Cluster'
    type: string
    default: 'demo'
  - name: nodeVmSize
    displayName: 'VM Size for the Nodes'
    type: string
    default: 'Standard_D2s_V3'
    values:
      - 'Standard_D2s_V3'
      - 'Standard_DS2_v2'
      - 'Standard_D4s_V3'
      - 'Standard_DS3_v2'
      - 'Standard_DS4_v2'
      - 'Standard_D8s_v3'
  - name: nodeCount
    displayName: 'The number of nodes'
    type: number
    default: 3

- ${{ each item in parameters }}:  
   - pwsh: | 
       $obj = '${{ convertToJson(item.value) }}' | ConvertFrom-Json 
       $value = ($obj | ConvertTo-Json -Compress) 
       if($obj.GetType().Name -eq "String") { 
         $value = $obj 
       }  
       Write-Host "##vso[task.setvariable variable=parameters.${{ item.key }}.value;]$value"
     displayName: "Create Variable ${{ item.key }}"

- task: FileTransform@2
  displayName: "Transform Parameters"
  inputs:
    folderPath: '$(System.DefaultWorkingDirectory)'
    xmlTransformationRules: ''
    jsonTargetFiles: 'deploy.parameters.json'

example of the PowerShell step with optional part to ignore unwanted parameters using notIn. 

- ${{ each item in parameters }}:  
  - ${{ if notIn(item.key, 'myunwantedparameter') }}:
    - pwsh: | 
        $obj = '${{ convertToJson(item.value) }}' | ConvertFrom-Json 
        $value = ($obj | ConvertTo-Json -Compress) 
        if($obj.GetType().Name -eq "String") { 
          $value = $obj 
        }  
        Write-Host "##vso[task.setvariable variable=parameters.${{ item.key }}.value;]$value"
      displayName: "Create Variable ${{ item.key }}"

Technique #3

Use a template file and PowerShell to create the parameters JSON file as a whole with all the parameters.

Note: Probably best used from a template file with the required parameters, however excludeParameters allows you to ignore ones you don’t need

createParametersFile.yml

parameters: 
- name: paramsJson 
  type: string 
- name: excludeParameters
  type: object
  default: []
- name: parameterFilePath
  type: string
  default: main.parameters.json
steps: 
- pwsh: | 
    $obj = '${{ parameters.paramsJson }}' | ConvertFrom-Json -AsHashtable
    $excludeList = '${{ convertToJson(parameters.excludeParameters) }}' | ConvertFrom-Json -Depth 10
    $header = [ordered]@{ 
      schema = "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#" 
      contentVersion = "1.0.0.0" 
      parameters = @{} 
    } 
    $valueObject = New-Object -TypeName PsObject 
    foreach ($item in $obj.GetEnumerator()) 
    {
       if ($item.Name -notin $excludeList) {
           $value = @{ 
             value = $item.Value 
           }
           Add-Member -InputObject $valueObject -MemberType NoteProperty -Name $item.Name -Value $value
       }
    } 
    $header.parameters = $valueObject 
    Set-Content ${{ parameters.parameterFilePath }} ($header | ConvertTo-Json -Depth 10)

Call the template:

parameters:
- name: tags
  displayName: 'Tags'
  type: object
  default:
     Environment: "prod"
     Resource: "AKS"
     Project: "Demo"
 - name: clusterName
    displayName: 'Name of the AKS Cluster'
    type: string
    default: 'demo'
  - name: nodeVmSize
    displayName: 'VM Size for the Nodes'
    type: string
    default: 'Standard_D2s_V3'
    values:
      - 'Standard_D2s_V3'
      - 'Standard_DS2_v2'
      - 'Standard_D4s_V3'
      - 'Standard_DS3_v2'
      - 'Standard_DS4_v2'
      - 'Standard_D8s_v3'
  - name: nodeCount
    displayName: 'The number of nodes'
    type: number
    default: 3

- template: createParametersFile.yml
  parameters:
     paramsJson: '${{ convertToJson(parameters) }}'
2
Azure Pipelines, DevOps

Optional Object/Array Checks in Azure Pipelines

Posted on by Mark Johnson

Sometimes when building Azure Pipelines in YAML you want to create a template with optional parameters and then test those parameters have a value.

For a string value this is straight forward e.g.

parameters:
- name: project
   type: string
   default: ''

${{ if eq(parameters.project, '') }}: 
or 
${{ if ne(parameters.project, '') }}:

When it comes to objects or arrays, creating the optional parameter is still straight forward e.g.

parameters:
- name: projects
   type: object
   default: []
- name: envs
   type: object
   default: {}

However, testing if they are empty is not quite so simple as you cannot just do an equality test like with string parameters e.g.

${{ if eq(parameters.envs, {}) }}: 
or
${{ if ne(parameters.envs, {}) }}:

${{ if eq(parameters.projects, []) }}: 
or
${{ if ne(parameters.projects, []) }}:

To get around this we can do a bit of trickery using convertToJson to provide a string representation of the object or array e.g.

${{ if eq(convertToJson(parameters.envs), '{}') }}: 
or 
${{ if ne(convertToJson(parameters.envs), '{}') }}:

${{ if eq(convertToJson(parameters.projects), '[]') }}:
or
${{ if ne(convertToJson(parameters.projects), '[]') }}:

Now this technique all sounds good but what about a real world example where it might be useful

So, I have a shared .NET build template (build-code.yml) and for some projects the dotnet test run requires some environment variables. We can pass the required environment variables in using the normal YAML object syntax e.g.

- template: build-code.yml
  parameters:
    version: '6.0.x'
    envs:
      appEndpoint: $(appEndpoint)
      databaseEndpoint: $(databaseEndpoint)
      apiEndpoint: $(apiEndpoint)

These are then used in build-code.yml template e.g.

parameters:
- name: version
  type: string
  default: ''
- name: artifactName
  type: string
  default: 'code'
- name: publishWebProjects
  type: boolean
  default: true
- name: zipAfterPublish
  type: boolean
  default: false
- name: envs
  type: object
  default: {}

variables: 
  buildConfiguration: 'Release'

steps:
- task: UseDotNet@2
  displayName: 'Use .NET version'
  inputs:
    packageType: 'sdk'
    ${{ if ne(parameters.version, '') }}:
      version: '${{ parameters.version }}'
    ${{ if eq(parameters.version, '') }}:
      useGlobalJson: true
- task: DotNetCoreCLI@2
  displayName: 'Restore Packages'
  inputs:
    command: restore
    projects: '**/*.csproj'
- task: DotNetCoreCLI@2
  displayName: 'Build Projects'
  inputs:
    command: 'build'
    projects: '**/*.csproj'
    arguments: '--no-restore --configuration $(buildConfiguration)'
- task: DotNetCoreCLI@2
  displayName: 'Run Tests'
  inputs:
    command: test
    projects: '**/*Tests.csproj'
    arguments: '--no-restore --no-build --configuration $(buildConfiguration)'
  ${{ if ne(convertToJson(parameters.envs), '{}') }}:
    env:
      ${{ parameters.envs }}

If the environment variables are not needed for a project then they can be simply omitted e.g.

- template: build-code.yml
  parameters:
    version: '6.0.x'

I don’t like to use too many optional parts in a template as it can get very complex but for this usage it works quite well