Azure Pipelines provides a FileTransform Task for variable substitution in configuration files, so given an appsettings file like this:
{
"Logging": {
"IncludeScopes": false,
"LogLevel": {
"Default": "Debug",
"System": "Information",
"Microsoft": "Information"
}
},
"WeatherSettings": {
"DefaultTemperatureType": "Celsius",
"DefaultWindType": "MPH",
"DefaultTemp": 30,
"ShowTemp": true
}
}
We could create pipeline variables that allow changes to the nested values e.g.
variables:
WeatherSettings.DefaultTemperatureType: 'Fahrenheit'
WeatherSettings.DefaultWindType: 'KMH'
WeatherSettings.DefaultTemp: 12
WeatherSettings.ShowTemp: false
So we could create a basic pipeline for the substitution :
trigger: none
pool:
vmImage: 'windows-latest'
variables:
appsettingsfile: appsettings.json
WeatherSettings.DefaultTemperatureType: 'Fahrenheit'
WeatherSettings.DefaultWindType: 'KMH'
WeatherSettings.DefaultTemp: 12
WeatherSettings.ShowTemp: false
steps:
- task: FileTransform@2
displayName: "Transform Json"
inputs:
folderPath: '$(System.DefaultWorkingDirectory)/**/'
xmlTransformationRules: ''
jsonTargetFiles: '**/$(appsettingsfile)'
- bash: |
cat $(appsettingsfile)
displayName: "Show Json substitution"
Now, what if we wanted to add parameters to set the values when running the pipeline?
We can turn all of those variables into parameters right? Seems a reasonable idea.
trigger: none
pool:
vmImage: 'windows-latest'
parameters:
- name: WeatherSettings.DefaultTemperatureType
type: string
default: 'Fahrenheit'
- name: WeatherSettings.DefaultWindType
type: string
default: 'KMH'
- name: WeatherSettings.DefaultTemp
type: number
default: 12
- name: WeatherSettings.ShowTemp
type: boolean
default: false
variables:
appsettingsfile: appsettings.json
steps:
- task: FileTransform@2
displayName: "Transform Json"
inputs:
folderPath: '$(System.DefaultWorkingDirectory)/**/'
xmlTransformationRules: ''
jsonTargetFiles: '**/$(appsettingsfile)'
- bash: |
cat $(appsettingsfile)
displayName: "Show Json substitution"
Well that doesn’t work, the parameter values are not picked up by the FileTransform Task and so no substitution happens. So, what if we define the variables again and assign the values from the parameters.
trigger: none
pool:
vmImage: 'windows-latest'
parameters:
- name: WeatherSettings.DefaultTemperatureType
type: string
default: 'Fahrenheit'
- name: WeatherSettings.DefaultWindType
type: string
default: 'KMH'
- name: WeatherSettings.DefaultTemp
type: number
default: 12
- name: WeatherSettings.ShowTemp
type: boolean
default: false
variables:
appsettingsfile: appsettings.json
WeatherSettings.DefaultTemperatureType: ${{ parameters.WeatherSettings.DefaultTemperatureType }}
WeatherSettings.DefaultWindType: ${{ parameters.WeatherSettings.DefaultWindType }}
WeatherSettings.DefaultTemp: ${{ parameters.WeatherSettings.DefaultTemp }}
WeatherSettings.ShowTemp: ${{ parameters.WeatherSettings.ShowTemp }}
steps:
- task: FileTransform@2
displayName: "Transform Json"
inputs:
folderPath: '$(System.DefaultWorkingDirectory)/**/'
xmlTransformationRules: ''
jsonTargetFiles: '**/$(appsettingsfile)'
- bash: |
cat $(appsettingsfile)
displayName: "Show Json substitution"
That doesn’t work either, using nested parameter names threw an error “Key not found”. OK so what if we change the parameters to not use nested names.
trigger: none
pool:
vmImage: 'windows-latest'
parameters:
- name: DefaultTemperatureType
type: string
default: 'Fahrenheit'
- name: DefaultWindType
type: string
default: 'KMH'
- name: DefaultTemp
type: number
default: 12
- name: ShowTemp
type: boolean
default: false
variables:
appsettingsfile: appsettings.json
WeatherSettings.DefaultTemperatureType: ${{ parameters.DefaultTemperatureType }}
WeatherSettings.DefaultWindType: ${{ parameters.DefaultWindType }}
WeatherSettings.DefaultTemp: ${{ parameters.DefaultTemp }}
WeatherSettings.ShowTemp: ${{ parameters.ShowTemp }}
steps:
- task: FileTransform@2
displayName: "Transform Json"
inputs:
folderPath: '$(System.DefaultWorkingDirectory)/**/'
xmlTransformationRules: ''
jsonTargetFiles: '**/$(appsettingsfile)'
- bash: |
cat $(appsettingsfile)
displayName: "Show Json substitution"
Great!! this now works and provides the substitution. Ah! but the boolean values become a string and the casing is invalid JSON.
"WeatherSettings": {
"DefaultTemperatureType": "Fahrenheit",
"DefaultWindType": "KMH",
"DefaultTemp": 12,
"ShowTemp": "False"
}
At the time of writing this is a known issue with the FileTransform Task but there is a work around. Simply change the boolean variables to be a string.
parameters:
- name: DefaultTemperatureType
type: string
default: 'Fahrenheit'
- name: DefaultWindType
type: string
default: 'KMH'
- name: DefaultTemp
type: number
default: 12
- name: ShowTemp
type: string
default: 'false'
This now has the correct output and adds the boolean value in the JSON.
"WeatherSettings": {
"DefaultTemperatureType": "Fahrenheit",
"DefaultWindType": "KMH",
"DefaultTemp": 12,
"ShowTemp": false
}
It’s good this is now working but it seems a bit excessive to add duplicate variable for each parameter in order to successfully get this to work. What could we do to improve that?
Well one thing we can do is to use a loop to turn all of the parameters into variables at runtime.
trigger: none
pool:
vmImage: 'windows-latest'
parameters:
- name: WeatherSettings.DefaultTemperatureType
type: string
default: 'Fahrenheit'
- name: WeatherSettings.DefaultWindType
type: string
default: 'KMH'
- name: WeatherSettings.DefaultTemp
type: number
default: 12
- name: WeatherSettings.ShowTemp
type: string
default: 'false'
variables:
appsettingsfile: appsettings.json
steps:
- ${{ each item in parameters }}:
- bash: |
echo "##vso[task.setvariable variable=${{ item.key }}]${{ item.value }}"
displayName: "Create Variable ${{ item.key }}"
- task: FileTransform@2
displayName: "Transform Json"
inputs:
folderPath: '$(System.DefaultWorkingDirectory)/**/'
xmlTransformationRules: ''
jsonTargetFiles: '**/$(appsettingsfile)'
- bash: |
cat $(appsettingsfile)
displayName: "Show Json substitution"
Note: The Azure Pipelines editor shows the loop underlined and reports an error “The first property must be task”.
But when running validate.
The result is OK. Hopefully this issue will get resolved in the future and not show as an error in the UI.
So the loop worked and provided all of the variables to be substituted. We could leave it there as it works, but we could also use some of the other parameter properties such as displayName and values to provide a nicer configuration.
trigger: none
pool:
vmImage: 'windows-latest'
parameters:
- name: WeatherSettings.DefaultTemperatureType
displayName: Temperature Type
type: string
default: 'Celsius'
values:
- 'Celsius'
- 'Fahrenheit'
- name: WeatherSettings.DefaultWindType
displayName: Wind Type
type: string
default: 'MPH'
values:
- 'MPH'
- 'KMH'
- name: WeatherSettings.DefaultTemp
displayName: Temperature
type: number
default: 30
- name: WeatherSettings.ShowTemp
displayName: Show Temperature
type: string
default: 'true'
values:
- 'true'
- 'false'
variables:
appsettingsfile: appsettings.json
steps:
- ${{ each item in parameters }}:
- bash: |
echo "##vso[task.setvariable variable=${{ item.key }}]${{ item.value }}"
displayName: "Create Variable ${{ item.key }}"
- task: FileTransform@2
displayName: "Transform Json"
inputs:
folderPath: '$(System.DefaultWorkingDirectory)/**/'
xmlTransformationRules: ''
jsonTargetFiles: '**/$(appsettingsfile)'
- bash: |
cat $(appsettingsfile)
displayName: "Show Json substitution"
In the Azure DevOps UI we can see the parameters have nice names instead of the nested ones and we can choose expected values.
I found this technique works really well and have already used it in a pipeline. You can also use a condition to only create a variable for parameters that starts with or contains etc. e.g.
- ${{ each item in parameters }}:
- bash: |
echo "##vso[task.setvariable variable=${{ item.key }}]${{ item.value }}"
displayName: "Create Variable ${{ item.key }}"
condition: startswith('${{ item.key }}', 'WeatherSettings')
Coding With Taz 