Monday, November 28, 2016

The Self-Healing Data Center Part 1: Using vR Ops with vRO to Automatically Remediate Alerts

If you are a user of vR OPs, you know that it can monitor your infrastructure, server OS, applications and more.  But as this commercial suggests, monitoring is only part of the answer.  Wouldn't it be much better to have vR Ops attempt some simple fixes before giving up and calling for human intervention?

In this blog post series, I will explain how to activate a vRealize Orchestrator workflow based on a vR Ops alert to fix an issue instead of just alerting you.  First some background.


Currently, vR Ops alerting can trigger notifications via email, SNMP trap or REST notification.  The REST notification plugin allows you to interact with REST APIs from practically any third-party system - almost!

Setting up an Alert notification rule in vR Ops
The truth is the REST Notification Plugin is not very robust and does not provide a lot of control over format of the REST call.  Basically, you plug in the URI and some credentials and vR Ops will try a PUT and POST against that URI.  By the way, it will append the alert ID onto the end of the URI.

Adding a REST Notification Plugin instance in vR Ops.  Note the limited options available; there is nowhere to format the request body or control the method used.

What this means is that unless your REST API is expecting the request in the way vR Ops would like to send it - it will fail. 

For example, Orchestrator has a nice REST API.  To launch a workflow via that API, I would send:

curl -X POST -H "Authorization: Basic dmNvYWRtaW46dmNvYWRtaW4=" -H "Accept: application/json" -d '{
  "parameters": [
    {
      "value": {},
      "name": "",
      "type": "",
      "description": "",
      "scope": "",
      "updated": false
    }
  ]
}' "https://10.140.45.13:8281/vco/api/workflows/a0b8b820-b8c9-454e-b128-c4eabb9d0015/executions"

As you can see, Orchestrator expects a body for this request that includes any inputs.  By the way, Orchestrator expects this payload even if there are no inputs.

So, my vR Ops REST call to Orchestrator will fail because it is not making a valid request.  How can we fix this?

One way is to translate the REST call from vR Ops into a REST call that can be consumed by Orchestrator.  

Enter the Webhook Shims for Log Insight and vRealize Operations!  This is a neat solution, originally crafted for Log Insight Webhooks (which suffers from similar limitations as vR Ops REST Notification Plugin).  It has been expanded to include capability for vR Ops.  This diagram shows how it works:


Basically, the Orchestrator shim takes the simplistic input from vR Ops REST notification and rebuilds the request in the format expected by Orchestrator.

In the next blog post, I will show you how to implement this solution to automatically attempt remediation based on alert notifications from vR Ops.

No comments:

Post a Comment