Networker automated recovery testing using the REST API - introduction

One feature that is lacking from Networker, compared to some of its competitors, is the built-in automatic recovery testing. However, when there's an API, there is a way.  Networker's REST API is not perfect, but it allows the backup administrator to perform queries about Networker resources (objects).

As my workload is going up, I realize that one of the tasks that I tend to skip the most is the periodical recovery tests.  Don't forget that a a backup that is not tested should be considered non-successful. I also found out that my recovery tests were not diverse enough.

When I started this project, I knew a little bit about REST APIs, and nothing about JSON processing. With the Networker REST API documentation, and the help of a friend and Networker Support staff, I was able to create HTTP queries with Postman,  cURL and jq. Once I got the queries that I needed, I put them in a bash script that would somehow select one backup, and then restore it.

My first attempt was to create a recover for an agent-based, filesystem backup.

My second attempt was to create a recover for a VMWare virtual machine. In this script, I perform added a few features:
  • The name of the client to recover it passed as the first argument
    • In my filesystem recover script, the client ID was hard-coded in the script
  • The backup that is used to perform the recover is selected randomly from the last 30 backups
  • My plan is to add a parameter or randomness to select the clone or the backup copy
For now the goal of this script is to perform a restore type of "New", meaning that a new virtual machine will be created and the data will be recovered there.  There are other types of recovery for virtual machines, but this is the less dangerous and the most accessible.

I'll share the scripts when they're complete.

Note : I discovered that the access management for the user that is used to execute the REST calls is not done at the REST level. Check your daemon.raw log for this kind of errors:

nsrjobd SYSTEM critical User nsrtomcat does not have the privileges required to start the requested job. The required privileges are: ('Recover Local Data' AND 'Recover Remote Data') for Recovery Operation OR ('Operate NetWorker') for Other Operation. The job requested to be started is: nsrvproxy_recover.

This article:
The Dell EMC Networker REST API Reference Guide for my version
Inspiration: this blog post from the data protection expert Preston De Guise:
Networker API triage guide:

Link to the next article (that contains a link to the Github repo I created for tools for Networker automated recoveries):  


Popular posts from this blog

General linux performance troubleshooting

Asterisk works under OpenVZ (no zaptel)