Azure Batch – Pool Auto scale forumulas

Just playing around with some Azure Batch Auto-scaling formulas.
See the Azure documentation here : https://azure.microsoft.com/en-gb/documentation/articles/batch-automatic-scaling/

Time based – Static

Between 8am and 6pm, Monday – Wednesday (0=Sunday) 5 nodes will be allocated to the pool.

Task based – Dev/Economical

Restrict capacity to 0 unless there is a waiting task – then allocates a single vm node to take the job.
This will built up to 5 minutes of latency before the pool autoscales, not including node boot time, and any pre-tasks the pool has associated. In my case this took up to 12 minutes before the scale occurred, originally going over my coded task schedulers timeout of 10 minutes.

Combination of task and time

On Monday (8am-6pm) ensure a minimum capacity of 1. When there is a waiting task in the queue, increase the pool capacity by 1 node.

Hard limit

As above, but ensuring that the maximum pool size never exceeds 10.

Azure Batch Start task – JRE windows | Java Runtime Environment

I’ve had to set Azure Batch up for a Java application, on Windows (go figure 🙂 ).

The first thing you need to do is get the Java Offline Installer.
http://java.com/en/download/manual.jsp

To launch the JRE install sliently, you’ll want to pass the /s flag. This only works with the offline installer.
Ref: http://docs.oracle.com/javase/8/docs/technotes/guides/install/windows_installer_options.html

Upload the installer to an Azure storage account, and note the public url to it.

Go to your Azure Batch account in the portal, click on the compute pool that you want to add the start task for.
Click resources, and enter the Azure Storage URL of the installer along with a destination path.

NB: Resource Paths are relational. Just enter the name of the file you want created, and it’ll be created here “C:\user\tasks\startup\wd\jre-8u101-windows-x64.exe”
EG;
Bad | “d:\jre” : “The resource file path is not valid”
Bad | “d:\jre\jre-8u101-windows-x64.exe” : “The resource file path is not valid”
Good | “jre-8u101-windows-x64.exe”

You should choose the options to “Run Elevated” and to “Wait for Success”.
AzureBatchStartupTask

When you add a node to the pool, you’ll find that Java is installed.
JavaInstalledBatch

Logic Apps – Json Schema Verify

Logic Apps has got a really handy trigger : Request/Response.
It provides a public URL that you can post to, and takes an optional Json Schema.  The benefit of providing a Json schema is purely that the fields defined in the schema are then accessible throughout the Logic App workflow for interaction with other actions.
This is awesome, it really makes the Visual aspect of the designer work well –  but being picky – there’s no built in validation of the Json Request against the schema you provide.

I’ve got a pretty well defined Json schema that uses

  • Required Fields
  • Typed fields (string/integer/etc)
  • Regex patterns for strings

As such i’m pretty keen to first find out if the Json in the request is actually valid.  If it’s not – I can return a 400 pretty quickly to the caller and save some valuable cash on my Logic Apps actions (Logic Apps Price Model).

The solution I went for was to wrap the Newtonsoft Json Schema library in an Azure Function.

First complication is passing in two parameters to an Azure function (the json from the Request body and the Json Schema).  Both are needed in order to perform a schema validation, and the Azure function can’t reach out to Logic Apps.  Switch to Code View in the Logic App designer and use something like this;

Apart from that, the coding of the Azure function is pretty easy.
It’s in GitHub here https://github.com/Gordonby/AzFunctions/tree/master/JsonValidate, but here’s a snapshot of the files.

A simple implementation of this can be seen below.  A Request trigger which passes the body to my Function App, the Response then just relays in information provided by the output of the app.JsonSchemaValidate