Cost optimising your Yaml Azure DevOps pipeline
In one of my pipelines, I’ve got some .NET Framework code that I’ve wrapped in a Container. One thing about Windows Containers that gets a little annoying is that you need to match the Container base image to the Windows OS that you’re building and running it on.
My image is based on mcr.microsoft.com/dotnet/core/runtime:2.2-nanoserver-1809 which necessitates me self hosting a Windows VM Azure DevOps build agent.
The Problem
Naturally i try and keep my Azure subscription costs low, so i have a Shutdown policy set for the VM so it deallocates at 7pm each night. This does leave me with the problem of builds queuing up during the day until i remember to turn the VM on. What i need is a smart way of the Pipeline starting the VM first.
Solutions
I started exploring a solution using an Azure DevOps Agentless job (in order to achieve maximum savings).
It would call out to an existing piece of code i have in Azure Functions which starts a VM based on values in the Request Body, Managed Identity and the right RBAC roles being set up. https://github.com/Gordonby/VMControlFunctions
The issue was that the Agentless Azure Functions task and the REST API task have a hard timeout of 20 seconds, which causes problems when you’re waiting for code to run that starts a VM.
The complex answer to this problem is to use a Callbacks, or to use a different codebase that facilitates not waiting on a response from the Azure RM API.
The much simpler answer is to use a hosted agent to run some Azure Powershell.
- stage: dev displayName: Dev Environment jobs: - job: StartBuildVM pool: vmImage: 'ubuntu-latest' steps: - task: AzurePowerShell@4 displayName: 'Start Windows Build VM' inputs: azureSubscription: 'mysub' ScriptPath: scripts/startwindowsbuildvm.ps1 azurePowerShellVersion: LatestVersion
$rg ="DevOpsBuildAgents" $vmname ="Win2019Builder" $VM = Get-AzVM -ResourceGroupName $rg -Name $vmname $VM | Start-AzVM
After the PowerShell runs, you can then use your specific Agent Pool to build and deploy the container.
- job: build pool: name: Windows Pool demands: Agent.OS -equals Windows_NT steps: - task: Docker@0 displayName: 'Build a container image' inputs: azureSubscription: 'mysub' azureContainerRegistry: '' dockerFile: HamWinTimer/Dockerfile imageName: '$(Build.DefinitionName):$(Build.BuildId)' includeLatestTag: true - task: Docker@0 displayName: 'Push a container image' inputs: azureSubscription: 'mysub' azureContainerRegistry: '' action: 'Push an image' imageName: '$(Build.DefinitionName):$(Build.BuildId)' includeLatestTag: true
My Azure Devops project code this code is open, if you’re wanting a look at the resulting Pipeline. It’s done using the new Multi-Stge YAML feature, although it started life as a Classic build pipeline. https://gordonbyers.visualstudio.com/Ham/_build?definitionId=9&_a=summary