Quick Tip: Execute a Single State ID from a Named SaltStack Module

Last week, I was working on enabling SaltStack Beacons in our environment to notify my team if one or more of a number of important system configuration files is modified and part of the configuration process involves defining your chosen beacon(s) in the Minon’s configuration file. Manually defining becons on a single Minion is a pretty painless process, but manually defining the same beacons across our entire environment wasn’t something I wanted to do. Thankfully, we manage our Minion configruation file with Salt (yes, Salt can salt itself…with a few caveats) so I was able to define the Beacon configruation once and push it out to our entire environment. However, I didn’t want to execute the entire module the managed file is part of.

Continue reading “Quick Tip: Execute a Single State ID from a Named SaltStack Module”

Configure SaltStack Reactor

In a previous post I explained how to send a message to a specific Slack channel from a Salt state. Today, I want to tie that into another important concept of SaltStack: the Event Reactor (or Reactor for short). Reactor is a somewhat advanced featrue of Salt, but it opens up untold possibilities once you understand how it works. At a high level, Reactor responds, or reacts, to certain events received on the event bus. For example, let’s say you want to send a Slack message to a speciifc channel if a critical service fails to notify operations staff. That’s totally possible with Reactor. What if you want to send a Slack message then try to restart the failed service? Not a problem. In this post we’ll configure Reactor on the Salt master to listen for a specific event that indicates Apache (httpd) has failed then write a Reactor State to send a Slack message to our #alerts Slack channel and, finally, attempt to restart Apache.

Continue reading “Configure SaltStack Reactor”

Dockerize an Existing WordPress Installation

Docker is an unbelievably popular technology right now. It’s extremely flexible and can be used at every stage of an application’s lifecycle. In fact, a Docker container and the application running inside it can move together through the application lifecycle all the way to production. In this post, we’re going to cover some of the core concepts of Docker by looking at how you can migrate an existing WordPress installation into Docker (or “Dockerize” it). For the purposes of this article, I’ll be creating Dockerized copy of this very blog. Before we get started, Let’s get started!

Continue reading “Dockerize an Existing WordPress Installation”

How-to: Harden SSH Authentication with Google Authenticator

A long, long time ago (in this very galaxy) I wrote a post on how to Harden SSH on a Linux instance with Multi-Factor Authentication which focused on using a Yubikey to add an extra layer of security to the traditional SSH login process. Today though, I’d like to follow up on that post and examine the use of OTP (One-Time Password) tokens generated via an app to secure SSH login in much the same way as the Yubikey. The primary benefit to using a software-based token as opposed to the physical Yubikey token is that you don’t have to incur the expense of purchasing a Yubikey. We’ll be using Google Authenticator and CentOS, but the same principles apply to virtually all Linux distribution that use PAM-based authentication. It’s also important to note that other apps such as Authy and 1Password can be used to generate the OTP tokens. We’re using Google Authenticator due to it’s ease of use and ubiquity. Let’s get started!

Continue reading “How-to: Harden SSH Authentication with Google Authenticator”

Quick Tip: Locate the Temporary Password in a MySQL Community Server Installation

I was working on a project at work last week which required me to use the MySQL Community Server distribution of MySQL. This was my first time ever using this variant of MySQL Server, but I didn’t see any fundamental differences between it and more mainstream versions like MariaDB, etc. Unfortunately, I would soon be proven wrong. As I was going through my usual database server deployment checklist, I started the mysql_secure_installation script and was unable to use it to set the password for the root user. This seemed strange to me since, to my knowledge anyway, MySQL shipped with no root password set and relied on the user to set one. Apparently, that’s not the case with MySQL Community Server. Instead, MySQL Community Server generates a temporary password for the root user and applies it during initial start-up. After a bit of Googling I found that the generated password is logged to /var/log/mysqld.log so we can use grep to quickly figure out what it is. To determine the default root password generated by MySQL Community Server during initial start-up run the following command:

Then run mysql_secure_installation and follow the prompts as you normally would.


Automatically Create an OmniFocus Task with AppleScript and BASH

Thanks to the Mac Power Users podcast, I discovered OmniFocus about a year and a half ago and it changed my lifeSeriously. I could (and very well may) write an entire post about my love for OmniFocus, but today I want to talk about mental friction and how I elevated a little of mine. Tools like OmniFocus are great for drastically reducing stress by allowing you to easily get all the stuff in your head into a trusted system, process it and take action on it as needed. If you’re anything like me though you put everything into your system and end up some tasks that, in some cases, repeat daily wether they’re actionable or not. The repeating task that prompted me to figure out how to automatically create an OmniFocus task was “Process action folder”. The concept of the action folder is simple: as things come into your system that need to be dealt with later in the day, they’re moved to the action folder and that folder is processed, well, later that day. I chose to put my action folder on Dropbox and having a universally accessible location where I could drop and continue with whatever I was doing while knowing it would get dealt with later was awesome! That is, until I discovered DevonTHINK.

Continue reading “Automatically Create an OmniFocus Task with AppleScript and BASH”

Send a Slack Message from a SaltStack Reactor State

My current side project at work is more than a little ambitious. In the past, we’ve had servers randomly “fall off” of our FreeIPA environment and stop authenticating for one reason or another and, after fixing what I honestly believe to be every possible client-side issue a FreeIPA-joined server can experience, I decided there had to be a better way. I’ve always been a huge fan of automation so I wanted a way to automatically detect a server that was failing to authenticate and fix it without any intervention on my part. I started with the quintessential Linux-based automation technology: shell scripting, but I ultimately ended up designing a solution based on SaltStack and SaltStack Reactors. While not yet complete, a few useful learnings have come out of my development work thus far. The learning we’re discussing today is, of course, sending a Slack message from a SaltStack Reactor State.

We aren’t technically a DevOps shop, but we’re steadily introducing more and more DevOps oriented tools and techniques, not the least of which is Slack. We love Slack and use it for team communication as well as alerting from Rundeck jobs so it made sense to send alerts from our Salt states to Slack as well. In my use case I wanted to send three different alerts: a start alert, a success alert or a failure alert. In today’s post, we’ll be looking at how to send the start notification, but the only thing that differs in the other two alerts is the message content. Let’s examine the code that makes this happen:

1. send_start_notification_to_slack_channel: – This line is the ID declaration and can be any unique string you choose.

2. local.slack.post_message: – This line declares the slack.post_message state. Notice local at the beginning of the line, local instructs the Minion running this Reactor State to run the slack.post_message state locally.

3. tgt: {{ data[‘id’] }} – This line sets the target for this state as the id of the Minion running this Reactor.

4. arg: – This line simply begins the list of arguments to the state.

5. #non-prod-alerts – This line indicates the Slack channel name to send the message to. Replace this with a channel in your team.

6. This is a test message sent by {{ data[‘id’] }} and brought to you by SaltStack. – This line contains the actual message body and can be any string you like. Notice that we use {{ data[‘id’] }} again in the middle of the string. {{ data[‘id’] }} will be replaced with the ID of the Minion running this Reactor so if, for example, server1.example.com ran this Reactor State the resulting message would read “This is a test message sent by server1.example.com and brought to you by SaltStack.”

7. saltstack – This line defines the username that will appear to be sending these messages. Replace this with a username that suits your needs.

8. <slack_API_key_goes_here> – Much as the description implies, your Slack API key goes here. Replace this placeholder with your Slack API key.

Whenever a Minion sends a correctly tagged event to the Master and the state is triggered, a message similar to the one shown below will be sent to the designated Slack channel.

That’s all there is to it! With a mere eight lines of code you’ve enabled your Salt Reactor States (and configuration states with a few minor changes) to notify your or your entire team via Slack. This could provide super useful as a way to notify someone when a long running state has completed or even function as a “poor man’s” monitoring system. In a future post I’ll bring the concept of Reactor States full-circle and discuss how to trigger them from Minions, but for now you can continue your salty foray into DevOps by reading over the Salt.States.Slack documentation.

— J

Goodbye 2016. Hello 2017!

There’s no denying 2016 was very, very good to me. I graduated from college, found my dream job and was able to remove a very negative influence from my life! I’ve never been one to rest on my laurels though, so as 2016 drew to a close I sat at Starbucks with my fancy coffee and MacBook Pro and pondered what I wanted from 2017. Early in 2016 I listened to an episode of Mac Power Users which featured Laura McClellan discussing her 3 words for 2016 and I was inspired to come up with my own in 2017. The concept of the 3 words centers around coming up with 3 words that function as overall themes for the year. From those themes, goals and a trajectory of sorts can start to form. For me, the 3 words, as well as the process of coming up with them, are super useful since I’m not only forced to reflect on the past year, but I also get to think very broadly about what I want from the next year. Once I know what I want from the year ahead at a high level I can then start thinking about goals, etc that map to those high level concepts.

Without further ado, here are my 3 words for 2017:

Healthy –  I’ve always struggled with weight loss. While I’ve lost considerable amounts of weight with various diets over the years, I’ve never managed to keep it off and I’ve actually gained weight with each diet. Today, I’m hovering around 260 pounds and I feel terrible. I joined LA Fitness last month, but I’ve only been twice. I tell myself I’m too busy most of the time to go to the gym and, though partially true, I can make and will make the time to go in 2017. With regular gym attendance and a decent diet I hope to lose 80 pounds or so and end the year around 180 pounds.

Wealthy – I’m lucky to be paid very well to do something I love. In fact, I actually look forward to going to work every day! While I make good money, I’m finding that I also spend quite a bit. In 2017, I want to become more fiscally responsible and think more about the purchases I’m making. If I can’t fully justify a potential purchase I’m going to put that money into savings and watch it grow throughout the year. I’ve been seriously considering buying a house in the next few years (damn you House Hunters!) so changing my spending habits and redirecting $10,000 – $15,000/year into savings would be a great start toward a down payment.

Wise –  I’m strange for a great many reasons (ask anyone who knows me personallly), but one of the biggest is that I actually like school and learning. I finished my Bachelor’s Degree in 2016 and I’ve sort of missed being in school since graduating in April. My job, quite generously, pays for professonal development and technical certifications so I plan to take advantage of that fact and learn Python while also “leveling-up” my Linux administration skills with an RHCE (Red Hat Certified Engineer) certification in 2017. I’ve been toying with the idea of Graduate School and an MBA, but that’s still a few years off.

I look forward to the end of 2017 when I can write the follow-up to this post and see how I did with the goals and concepts set forth here. So, what are you three (or more) words for 2017? Leave them in a comment below.

Quick Tip: Resolve Salt-Cloud EC2 Instance Provisioning Failure during Dependency Installation

I recently began learning SaltStack to compliment my knowledge of Puppet. I always like to use a new technology I’m learning to accomplish meaningful tasks related to whatever project I’m working on at the time and SaltStack, along with its cloud-oriented companion Salt-Cloud, seemed like they would be very helpful when migrating my personal website and Blog from Squarespace back to Amazon Web Services. Once I had deployed a salt-master server and configured it with the necessary packages and profiles, I attempted to deploy my first EC2 instance using Salt-Cloud and ran into the following error:

[bash]salt-cloud * ERROR: Failed to run install_amazon_linux_ami_deps()!!![/bash]
Continue reading “Quick Tip: Resolve Salt-Cloud EC2 Instance Provisioning Failure during Dependency Installation”

Harden SSH on a Linux instance with Multi-Factor Authentication

With stories like the recent iCloud hack popping up in mainstream media, security and privacy are becoming increasingly important to the average consumer. These topics however are even more important to system administrators, those of us entrusted with safeguarding sensitive data against an ever changing threat. As technology professionals we have access to a number of tools to assist in defending our networks and protecting private data, but one of the most powerful tools at our disposal is multi-factor authentication. In this post we will take a high level look at multi-factor authentication and discuss the implementation of one multi-factor authentication solution to secure SSH access on a Linux instance. Continue reading “Harden SSH on a Linux instance with Multi-Factor Authentication”