Azure Logic Apps, tips from the real world

About 2 years ago I wrote some PowerShell as a workflow function that could be used in Azure Automation. The purpose was simple, get a daily email informing me of yesterdays Azure spend and then predict my annual charge based on it. I had a colleague ask me about it, and it got me thinking – pretty much the entirety of my script could be replicated in an Azure Logic App. Here’s the script if you’re interested : http://gordon.byers.me/azure/estimating-your-annual-azure-bill/

Translating the PowerShell function into a Logic App wouldn’t really add much value to the end product, i’d still get an email… However it would be a good experience for eating the Logic App dog food and seeing where the capabilities fall short and I have to use some code. It also makes it very easy to customise with other connectors in the Logic Apps library. To that end, this post isn’t so much about Azure billing, but Logic Apps and the tips and tweaks that you might find useful when considering it for your own workloads.

So, lets start with the end result, then pick it apart.
LogicAppBilling

  1. The Trigger. A schedule. Once every day at approx. 9am.
  2. Initialise Variable. Setting an integer variable to the value for my Azure Enterprise Enrolment.
  3. Initialise Variable. Setting a string variable to the value for my EA Billing API key. (Ok, i’m cheating here as at the time Logic Apps doesn’t have string variables, but it’s coming soon!)
  4. Http call. Doing a GET to the EA billing api and asking for a summary of all usage reports.
  5. Blob create. Persisting the Summarised data from the api down to a container for future analysis
  6. Parse Json. Looking to what was returned in the Http call, and putting a Json Schema over the top to allow me to use this data in further Logic Apps.
  7. Parse Json. Filtering to the last month in the Array and putting a Json Schema over the top of it
  8. Blob create. Persisting the output to blob storage purely as part of the development process, so I can see what’s going on
  9. Http call. Doing a secondary GET to the EA billing api and asking for the detailed usage for the latest month
  10. Blob create. Persisting the detailed data down to blob storage for future analysis
  11. Azure function. At this point I need something to take a CSV file, clean it up a little, do some maths and just return me the sum of my usage in a currency unit
  12. Blob create. Persisting the output to blob storage purely as part of the development process, so I can see what’s going on
  13. Gmail send. Sending an HTML email with the monthly cost and predicted cost

So as it stands I managed to get quite far through the process without reaching out to Azure Functions. When I did have to use my own code, I created a Powershell Azure Function which let me keep all my previously written code for parsing…. RESULT! Nothing worse than rewriting something that works.
Here’s how that code looks;

Tips and Tricks

Ok, so it wasn’t all plain sailing writing the logic app. Here’s a few areas where I needed to do more than connect the dots with the visual designer.

Parsing Json
These are really cool little actions. They let you put a JSON schema on top of an Json that you’ve taken into the Logic App. Previously you’d have multiple Logic Apps talking to each other over Request/Response triggers. For ultimate ease, they’ll create the schema for you based on the json data, so make sure you’ve got this to hand. I found that using the Run History UI in Logic Apps pretty handy, as was persisting it down to blob storage and then grabbing it out.

Filtering to the last item in an array
Using a Json Parse action worked well again here, and allowed me to use the LAST function to quickly focus on what I wanted.
json-summary-filter


Concatenating implicitly in an Http action’s URI field

Ok, it didn’t like this much at all. I had to break into code mode and use the CONCAT function to combine the base URL with the one resulted from the HTTP call.
@{concat('https://ea.azure.com/',body('Grab Latest Months Summary')?['LinkToDownloadDetailReport'])}

Wrapping the output in Json
An easy one, but one worth mentioning, when passing data around between functions and API’s you’ll often have to start wrapping it in JSON.
powershellwrap

Type conversion and maths
My PowerShell function returns a string, which I then need to convert to an integer and do some basic maths on.
I used MUL to perform a multiplication, and FLOAT to convert the string to a float that I could do the MUL on.
You spent £@{body('Parse-Azure-EA-Detail-CSV')} yesterday.
If every day was like yesterday, your 12 monthly Azure Consumption would be £@{mul(365,float(body('Parse-Azure-EA-Detail-CSV')))} 

I’ll be creating an ARM template for this and putting it on GitHub soon, so if you’re wanting to get a daily email with your Azure spend you’ll be able to quickly and easily.

Logic Apps – Json Schema Verify

Logic Apps has got a really handy trigger : Request/Response.
It provides a public URL that you can post to, and takes an optional Json Schema.  The benefit of providing a Json schema is purely that the fields defined in the schema are then accessible throughout the Logic App workflow for interaction with other actions.
This is awesome, it really makes the Visual aspect of the designer work well –  but being picky – there’s no built in validation of the Json Request against the schema you provide.

I’ve got a pretty well defined Json schema that uses

  • Required Fields
  • Typed fields (string/integer/etc)
  • Regex patterns for strings

As such i’m pretty keen to first find out if the Json in the request is actually valid.  If it’s not – I can return a 400 pretty quickly to the caller and save some valuable cash on my Logic Apps actions (Logic Apps Price Model).

The solution I went for was to wrap the Newtonsoft Json Schema library in an Azure Function.

First complication is passing in two parameters to an Azure function (the json from the Request body and the Json Schema).  Both are needed in order to perform a schema validation, and the Azure function can’t reach out to Logic Apps.  Switch to Code View in the Logic App designer and use something like this;

Apart from that, the coding of the Azure function is pretty easy.
It’s in GitHub here https://github.com/Gordonby/AzFunctions/tree/master/JsonValidate, but here’s a snapshot of the files.

A simple implementation of this can be seen below.  A Request trigger which passes the body to my Function App, the Response then just relays in information provided by the output of the app.JsonSchemaValidate

XML to JSON data manipulation with schemas

Starting with an XML file and then working backwards… Old school data munging at it’s worst, and the tools/tips that I used.

Creating a XSD schema from XML
In Visual Studio 2015, open the XML file. Find the XML main menu and select Generate XSD.

Creating a JSON schema from XSD
This seems to be a bit of an issue at the moment. There’s an old BizTalk 2013 component that will create a json schema of a sort, but not a Rev4 one.
Unfortunately this seems to be best tackled manually at the moment.

Creating a JSON from XML
Here’s my process if you have the original XML.
Convert XML to JSON. I did this in visual studio using Newtonsoft JsonConvert.
Use JSON Lint/ to Validate and Format. You might also want to pull of any XML header info from the JSON at this point.
Look at the resulting JSON. If you’ve got properties beginning with “@” (they’re XML attributes), then you need to bin these characters off – I found it caused problems later.

Creating a JSON schema from JSON
http://jsonschema.net/#/

Verifying a JSON schema against JSON
http://jsonschemalint.com

Creating a Swagger definition file for a Logic App
http://gordon.byers.me/azure/creating-a-swagger-definition-for-an-azure-logic-apps-http-request-endpoint/

Creating a XSLT Map for an Azure Integration Account
Have a read about the Enterprise Integration Pack, Install the VS2015 extension https://aka.ms/vsmapsandschemas
Create a Biztalk Integration projects to be created then a Integration Map file which will allow you to visually map items between two XML schema files.
After building this project, an XSLT document is output into the bin folder.

Testing a Logic App HTTP Request Endpoint
https://chrome.google.com/webstore/detail/postman

Creating a swagger definition for an Azure Logic Apps Http Request endpoint

If you’ve created a Logic App with a HTTP Request trigger, then next logical thing to do is expose it for consumption (Like in Azure API Management).
Swagger is now the defacto way of describing API’s, and it makes sense that you’d want to create one for your logic app. I’m sure this will make it in there as a feature soon, but for the moment you need to roll your own.

The LogicAppsRequestSchema is exactly the same Json schema that you used in the HTTP Request Trigger action, so keep that handy. The text in bold are the areas you need to replace with your own.

I find that http://editor.swagger.io is the best tool for creating the Swagger files.
The validation is great (although it doesn’t like the default values for the parameters, which as far as i can gather are valid).

{
"swagger": "2.0",
"info": {
"title": "Friendly name for your logic app",
"description": "Some kind of description about what it does",
"version": "1.0.0"
},
"host": "aprodinstance.anazureregion.logic.azure.com:443",
"schemes": [
"https"
],
"basePath": "/workflows/yourworkflowid/triggers/manual",
"produces": [
"application/json"
],
"paths": {
"/run": {
"post": {
"consumes": [
"application/json"
],
"operationId": "Friendly text",
"summary": "Friendly text",
"description": "Friendly text",
"parameters": [
{
"name": "api-version",
"in": "query",
"description": "api-version",
"required": true,
"default": "2016-06-01",
"type": "string"
},
{
"name": "sp",
"in": "query",
"description": "sp",
"required": true,
"default": "%2Ftriggers%2Fmanual%2Frun",
"type": "string"
},
{
"name": "sv",
"in": "query",
"description": "sv",
"required": true,
"default": "1.0",
"type": "string"
},
{
"name": "sig",
"in": "query",
"description": "sig",
"required": true,
"default": "thesignaturepartinyourlogicapphttpurl",
"type": "string"
},
{
"name": "body",
"in": "body",
"description": "The json you want to submit",
"required": true,
"schema": {
"$ref": "#/definitions/LogicAppsRequestSchema"
}
}
],
"responses": {
"200": {
"description": "Logic Apps response",
"schema": {
"$ref": "#/definitions/LogicAppsResponse"
}
}
}
}
}
},
"definitions": {
"LogicAppsRequestSchema":
{
"type": "object",
"required": [
"Header",
"StartOfDuty"
],
"properties": {
"Obj1": {
"properties": {
"Prop1": {
"type": "string"
}
"required": ["rop1"],
"type": "object"
},
"Obj2": {
"properties": {
"Prop1": {
"type": "string"
}
"required": ["Prop1"],
"type": "object"
}
}
}
,
"LogicAppsResponse": {
"type": "object",
"properties": {
"success": {
"type": "boolean",
"description": "Just a simple response property. You could template return objects/ error messages etc."
}
}
}
}
}