Wget

Ah, I do feel good when I solve a complex problem with a simple tool. wget. This time, I wanted to take a backup of the database I have hosted somewhere. The only way to get these databases' backup files is to go the control panel of my hoster, cpanel. Usually, then I think about that, it is too late (as these files are the current databases, so if I just screwed something, these files are useless)

I then remembered that wget can send authentication, so I basically use wget in a cronjob that downloads the database files and put them in a folder named after the current day's name. This way, I have 7 days of backup history. I like that :).

Comments

Post a Comment

Popular posts from this blog

General linux performance troubleshooting

Here's a list of commands that you should execute and then share the output to someone who can help you figure out what resource is the bottleneck in your system. Note, it requires the  sysstat  and  procps  packages (ubuntu and RHEL and its derivatives): uptime vmstat 1 10 iostat -xN 2 10 mpstat -P ALL 3 10 pidstat 1 10 free -mw (or free -m, if your OS doesn't support -w)  uptime is to see the load averages on the system. vmstat is mostly used to tell if the system is swapping or not. If you see significant numbers in the 'si' and 'so' columns, your system is most likely swapping (using the hard drive as RAM), which usually slows performance a lot. iostat is mostly used to determine if your disk subsystem is not able to cope with the load. If you see one or more lines that shows 100 or close almost constantly, it is probably the case.  If it is your swap volume, you probably saw numbers in the 'si' and 'so' columns in the  vmsta
Read more

Networker automated recovery testing using the REST API - introduction

One feature that is lacking from Networker, compared to some of its competitors, is the built-in automatic recovery testing. However, when there's an API, there is a way.  Networker's REST API is not perfect, but it allows the backup administrator to perform queries about Networker resources (objects). As my workload is going up, I realize that one of the tasks that I tend to skip the most is the periodical recovery tests.  Don't forget that a a backup that is not tested should be considered non-successful. I also found out that my recovery tests were not diverse enough. When I started this project, I knew a little bit about REST APIs, and nothing about JSON processing. With the Networker REST API documentation, and the help of a friend and Networker Support staff, I was able to create HTTP queries with Postman,  cURL and jq . Once I got the queries that I needed, I put them in a bash script that would somehow select one backup, and then restore it. My first attempt
Read more

Networker automated recovery testing using the REST API - first script

I have worked a lot on my automated recovery script recently and finally got to a version that is a really good proof of concept and that I'm not scared to publish.  Plus, I have created a Gitub repo for documentation, issue management and, of course, Git features. The script is not perfect but it is even better than what I was planning when I wrote my first post on the topic: A lot of checks have been put in place to make sure all variables are provided, and in the right format There is an inline help with -h The script chooses a backup randomly amongst all the available backups, not only the 30 last ones It only requires bash, cURL and jq, on a machine that has access to the Networker REST API Only tested on RHEL 6 for now The script is not complete (I already created more than 10 issues to add features, improve current features or fix things), but it is a very good start for an organisation that wants to make recovery tests easier. The Github repo will also allow
Read more