Cost optimising your Yaml Azure DevOps pipeline

Cost optimising your Yaml Azure DevOps pipeline

2019, Nov 01    

In one of my pipelines, I’ve got some .NET Framework code that I’ve wrapped in a Container. One thing about Windows Containers that gets a little annoying is that you need to match the Container base image to the Windows OS that you’re building and running it on.
My image is based on mcr.microsoft.com/dotnet/core/runtime:2.2-nanoserver-1809 which necessitates me self hosting a Windows VM Azure DevOps build agent.

The Problem

Naturally i try and keep my Azure subscription costs low, so i have a Shutdown policy set for the VM so it deallocates at 7pm each night. This does leave me with the problem of builds queuing up during the day until i remember to turn the VM on. What i need is a smart way of the Pipeline starting the VM first.

Solutions

I started exploring a solution using an Azure DevOps Agentless job (in order to achieve maximum savings).
It would call out to an existing piece of code i have in Azure Functions which starts a VM based on values in the Request Body, Managed Identity and the right RBAC roles being set up. https://github.com/Gordonby/VMControlFunctions

The issue was that the Agentless Azure Functions task and the REST API task have a hard timeout of 20 seconds, which causes problems when you’re waiting for code to run that starts a VM.
The complex answer to this problem is to use a Callbacks, or to use a different codebase that facilitates not waiting on a response from the Azure RM API.
The much simpler answer is to use a hosted agent to run some Azure Powershell.

- stage: dev
  displayName: Dev Environment
  jobs:
  - job: StartBuildVM
    pool:
      vmImage: 'ubuntu-latest'
    steps:
    - task: AzurePowerShell@4
      displayName: 'Start Windows Build VM'
      inputs:
        azureSubscription: 'mysub'
        ScriptPath: scripts/startwindowsbuildvm.ps1
        azurePowerShellVersion: LatestVersion
$rg ="DevOpsBuildAgents"
$vmname ="Win2019Builder"
$VM = Get-AzVM -ResourceGroupName $rg -Name $vmname
$VM | Start-AzVM

After the PowerShell runs, you can then use your specific Agent Pool to build and deploy the container.

- job: build
    pool:
      name: Windows Pool
      demands: Agent.OS -equals Windows_NT

    steps:
    - task: Docker@0
      displayName: 'Build a container image'
      inputs:
        azureSubscription: 'mysub'
        azureContainerRegistry: ''
        dockerFile: HamWinTimer/Dockerfile
        imageName: '$(Build.DefinitionName):$(Build.BuildId)'
        includeLatestTag: true

    - task: Docker@0
      displayName: 'Push a container image'
      inputs:
        azureSubscription: 'mysub'
        azureContainerRegistry: ''
        action: 'Push an image'
        imageName: '$(Build.DefinitionName):$(Build.BuildId)'
        includeLatestTag: true

My Azure Devops project code this code is open, if you’re wanting a look at the resulting Pipeline. It’s done using the new Multi-Stge YAML feature, although it started life as a Classic build pipeline. https://gordonbyers.visualstudio.com/Ham/_build?definitionId=9&_a=summary

visual pipeline tasks