NSX Manager to login with a local account
Posted on by Caenotech

NSX Backup Check Script (Using the NSX Web API)

I was recently asked for a way to have a simple check and report on the last backup status for a global company with multiple VMware NSX managers.

For some reason their NSX Managers were not reporting the backup status via syslog to VMware vRealize Log Insight (vRLI) and even if it was, they only have one vRLI cluster per site and wanted one simple place to do their daily checks.

So let’s make an NSX backup check script, PowerShell and REST API to the rescue! … right?

So … no, you cannot get the backup status via REST API

Great.

But you can via the WebAPI!

Hurrah! Lets throw in some Invoke-WebRequest and get the data we need.

After some basic checks, I got the info I wanted – now I need to schedule it and have it run a short period after the backup window.

This part resulted in a path of trying to figure out a way to hold account passwords in a usable manner without them being written in clear-text anywhere because that’s just no good. There are a few different ways to do this, but they either tie it to one user profile and computer, or don’t work with the basic auth needed to run against NSX to get the data via webrequest. I will go into how I achieved that further down, but first, the web API to get backup status.

Interrogating NSX Web API

After some looking around, I discovered the following URL called via Invoke-WebRequest would give us the backup results:

Invoke-WebRequest -Uri https://[nsxmgr]/api/v1/cluster/backups/overview -Headers $Header

Now the big problem with Invoke-WebRequest is that you would have assumed that it would return any response status such as 403 Forbidden. Nope!

You don’t get any helpful error catching, it either works or bombs out. Not much good for an unattended script that you want to tell you about any issues.

So the best fix I came up with was using a try and catch

try { 
    $result = Invoke-WebRequest -Uri https://...
    }
catch {catchFailure}

I then created a function to run in the event of the failure which will ping the host to see if it’s online and if it is output the error, if it isn’t output that the host is unreachable.

if (Test-Connection -BufferSize 32 -Count 1 -ComputerName $nsxmgr -Quiet) {
        <error output>
    } else { <host offline output> } exit

Job jobbed, no more bombing out with red text.

Dealing with certificates

When you run Invoke-WebRequest against an NSX manager with self signed certificates you get the error "The Underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel"

The fix for this is to add in this code near the top of your script to resolve the error.

add-type @"
   using System.Net;
   using System.Security.Cryptography.X509Certificates;
   public class TrustAllCertsPolicy : ICertificatePolicy {
      public bool CheckValidationResult(
      ServicePoint srvPoint, X509Certificate certificate,
      WebRequest request, int certificateProblem) {
      return true;
   }
}
"@
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy

Password Management

Great stuff I now have a working Script, but ideally, I want it to be scheduled and unattended.

This is where I spun around for a while trying different ways to store credentials in a secure format, because passwords in plain text is uncool.

I initially was trying to use the encoded credentials modules but having little luck getting it passed as a header value, so bugged a colleague (@pauldavey_79) for some help and ideas from his many years of experience prodding APIs.

What we came up with was to take the username and password as an requested input via Read-Host and encode it in the Base 64 format required to pass via the header in Invoke-WebRequest and store that in a text file.

[System.Runtime.InteropServices.Marshal]::PtrToStringAuto([System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($pass))
    $userpass  = $username + ":" + $password

    $bytes= [System.Text.Encoding]::UTF8.GetBytes($userpass)
    $encodedlogin=[Convert]::ToBase64String($bytes)
    
    Set-Content -Path $credPath -Value $encodedlogin

This worked a charm.

Handling the Outputs.

With this script I wanted to feed the output into a syslog server (vRealize Log Insight in this case) which would then send emails to the appropriate places and allow for a nice dashboard for quick whole estate checks.

In order to achieve this, I used the Add-Content command to append the data to a .log file which was monitored by the Log Insight Agent and sent off to the Log Insight Server.

if($LatestBackup.success -eq $true){ 
  Add-Content -Path $exportpath -Value "$timestamp [INFO] $NSXMGR - Last backup successful. Start time $start End time $end"
} else{ 
  Add-Content -Path $exportpath -Value "$timestamp [ERROR] $NSXMGR - Last backup failed $start $end"

This gives us a nice syslog formatted output which can be easily manipulated within Log Insight. Hurrah.

One thing to note is that the NSX WebAPI returned the start and end times in the usual unix format, so I needed to convert that to a more suitable human readable date, that was done with the line:

 $var = (get-date 01.01.1970).AddSeconds([int]($LatestBackup.end_time/1000))

I also needed to get my try-catch error collector to output the error messages in the same format so that was done as so:

Add-Content -Path $exportpath -Value "$timestamp [ERROR] $NSXMGR - $_"

Pulling all of that together we get the final script which can be used as a skeleton for any future work required. A few of them will be posted at a later date.

The Final NSX Backup Check Script

I have put all the variables at the top and the script is designed to be run in a folder and to have another separate folder with the logs. This was done in order to manage multiple scripts logging to the same location
eg:
c:\scripts\NSXBackupCheck\NSXBackupCheck.ps1
c:\scripts\Logs\NSXBackupCheck.log

param ($nsxmgr)

$curDir = &{$MyInvocation.PSScriptRoot}
$exportpath = "$curDir\..\Logs\NSXBackupCheck.log"
$credPath = "$curDir\$nsxmgr.cred"
$scriptName = &{$MyInvocation.ScriptName}

add-type @"
   using System.Net;
   using System.Security.Cryptography.X509Certificates;
   public class TrustAllCertsPolicy : ICertificatePolicy {
      public bool CheckValidationResult(
      ServicePoint srvPoint, X509Certificate certificate,
      WebRequest request, int certificateProblem) {
      return true;
   }
}
"@
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy

function catchFailure {
    $timestamp = (Get-Date).ToString("yyyy/MM/dd HH:mm:ss")
    if (Test-Connection -BufferSize 32 -Count 1 -ComputerName $nsxmgr -Quiet) {
        Add-Content -Path $exportpath -Value "$timestamp [ERROR] $NSXMGR - $_"
    }
    else {
        Add-Content -Path $exportpath -Value "$timestamp [ERROR] $NSXMGR - Host Not Found"
    }
exit
}

if (!$nsxmgr) {
    Write-Host "please provide parameter 'nsxmgr' in the format '$scriptName -nsxmgr [FQDN of NSX Manager]'"
    exit
    }

if (-Not(Test-Path -Path  $credPath)) {
    $username = Read-Host "Enter username for NSX Manager" 
    $pass = Read-Host "Enter password" -AsSecureString 
    $password = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto([System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($pass))
    $userpass  = $username + ":" + $password

    $bytes= [System.Text.Encoding]::UTF8.GetBytes($userpass)
    $encodedlogin=[Convert]::ToBase64String($bytes)
    
    Set-Content -Path $credPath -Value $encodedlogin
}

$encodedlogin = Get-Content -Path $credPath

$authheader = "Basic " + $encodedlogin
$header = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$header.Add("Authorization",$authheader)

try{
    $result = Invoke-WebRequest -Uri https://$nsxmgr/api/v1/cluster/backups/overview -Headers $Header -UseBasicParsing
    if($result.StatusCode -eq 200) {
        $nsxbackups = $result.Content | ConvertFrom-Json
        $LatestBackup = $nsxbackups.backup_operation_history.cluster_backup_statuses
        $start = (get-date 01.01.1970).AddSeconds([int]($LatestBackup.start_time/1000))
        $end = (get-date 01.01.1970).AddSeconds([int]($LatestBackup.end_time/1000))
        $timestamp = (Get-Date).ToString("yyyy/MM/dd HH:mm:ss")
        if($LatestBackup.success -eq $true){ 
            Add-Content -Path $exportpath -Value "$timestamp [INFO] $NSXMGR - Last backup successful. Start time $start End time $end"
        } else{ 
            Add-Content -Path $exportpath -Value "$timestamp [ERROR] $NSXMGR - Last backup failed $start $end"
        }
    }
 }
catch {catchFailure}

Overview

The final script above can be altered to be used as a skeleton for any other Invoke-WebRequest APIs as well as simply being adapted for REST API. I will be following up this post with further updates to this script using RESTAPI and also an adaption to use PowerCLI which required a different credential store.

The REST API Script can be found here: NSX Alarm Check Script