Posts

Tips for being a better system administrator

Here are a few tips that I have discovered or implemented during my career, that can help anyone get better results: Follow the 3-2-1 backup strategy  or better 3 copies of the data 3 different media 1 copy offsite Test your backups regularly. Ideally, automate your recovery tests. Use Whatever-as-code as much as you can. IaC is the first that comes to my mind, but using code to define objects and their properties have several advantages: Auto-documentation of changes Ability to easily rollback (using source code management like Git) Makes it easier for standardization an compliance Ansible, Chef, Puppet, Salt, Terraform, Pulumi are good examples Maintain a changelog of all the major changes in your infrastructure (that isn't already "documented" because you're using IaC). Keep track of your hardware inventory and plan your renewals and warranty extension purchases. Build a spreadsheet with all the components that you manage and make sure that everyone on the sysadmi...

CentOS6 - End of life - How to install packages?

 CentOS 6 is EOL (End of life) since November 2020.  How to install packages now that the default CentOS repos have been emptied? The script in this Github repo is very helpful for that:  https://github.com/EngineeredVirus/CentOS6-Repository-Fix . Have fun!

NRPE troubleshooting

 Hi,  To make nrpe daemon log plugins output, add this to the command definition: 2>&1   Example:    # 'check_local_mailq' command definition define command{         command_name    check_local_mailq         command_line    /usr/bin/sudo /usr/lib64/nagios/plugins/check_mailq -w 1 -c 2 > /tmp/nagiosdebug 2>&1         } Of course, make sure the user running nrpe or the plugin can write to the file ( /tmp/nagiosdebug in this case). Then: # tail -f /tmp/nagiosdebug sudo: sorry, you must have a tty to run sudo I just added this in my sudo configuration # Required for check_mailq Defaults:nagios !requiretty   And now it works! Important: once you're done debugging, remove the redirection, so that Nagios can get access to the output. Otherwise, you'll see "null" in the output instead of something like: OK: mailq i...

Symfony - clear cache

I was a bit surprised when I learnt that the only way to clear the cache on a Symfony 4 system is to run a local command on the system, as I don't want anyone to log into a production server (deployment is automated). And when you run the command, you must run it as the user that your web server is running under.  For distros of the Red Hat family, this user is apache by default.  For the Debian family, it is www-data. Here is how I did it.  Let's say that the user that is used for automated deployment is 'user123', here's what I did: Create a file in the /etc/sudoers.d directory (example: /etc/sudoers.d/user123_bin_console) Put this line in the file: user123 ALL=(apache) NOPASSWD: /path/to/bin/console Once this file is in place, you can log into the system and execute the sudo -l command to show what commands this user is allowed to execute via sudo and make sure the command you specified in the newly created file is present To clear the cache, execute this command...

Networker automated recovery testing using the REST API - first script

I have worked a lot on my automated recovery script recently and finally got to a version that is a really good proof of concept and that I'm not scared to publish.  Plus, I have created a Gitub repo for documentation, issue management and, of course, Git features. The script is not perfect but it is even better than what I was planning when I wrote my first post on the topic: A lot of checks have been put in place to make sure all variables are provided, and in the right format There is an inline help with -h The script chooses a backup randomly amongst all the available backups, not only the 30 last ones It only requires bash, cURL and jq, on a machine that has access to the Networker REST API Only tested on RHEL 6 for now The script is not complete (I already created more than 10 issues to add features, improve current features or fix things), but it is a very good start for an organisation that wants to make recovery tests easier. The Github repo will also allow...

Networker automated recovery testing using the REST API - introduction

One feature that is lacking from Networker, compared to some of its competitors, is the built-in automatic recovery testing. However, when there's an API, there is a way.  Networker's REST API is not perfect, but it allows the backup administrator to perform queries about Networker resources (objects). As my workload is going up, I realize that one of the tasks that I tend to skip the most is the periodical recovery tests.  Don't forget that a a backup that is not tested should be considered non-successful. I also found out that my recovery tests were not diverse enough. When I started this project, I knew a little bit about REST APIs, and nothing about JSON processing. With the Networker REST API documentation, and the help of a friend and Networker Support staff, I was able to create HTTP queries with Postman,  cURL and jq . Once I got the queries that I needed, I put them in a bash script that would somehow select one backup, and then restore it. My first attem...

For COVID-19, my server performance analysis course is now free!

Hi, My contribution to help people during this "stay-at-home" period: My course on Udemy is now free. You can enroll here . I hope that many of you will enroll and take the course. It's only about one hour long and have a lot of information about how to prevent, troubleshoot and solve performance problems in computing.

OTRS unmerge

Rough notes about un-merging tickets in OTRS. To delete the relationships, you simply delete relevant rows in the table named link_relation Afterwards, you re-assign articles to the right ticket (ticket_id) Regex .* Duplicated Parameter Name