Public key for epel-release-7-8.noarch.rpm is not installed

I was trying to run some ansible playbooks on my CentOS 7 Linux machine. I hit a failure because the version of ansible on the machine (1.9.4.0) was less than the minimum version required by the playbooks (2.1.0.0). yum install was seeing 1.9.4.0 as the latest.

It turns out what I needed was to pull a version of ansible from the EPEL repo rather than the default repo. yum repolist showed that the EPEL repo was already available on the machine, so I followed the instructions I found on the Internet: yum install ansible-2.1.0.0

The package was found and downloaded. But before the installation completed it hit an error:

Public key for epel-release-7-8.noarch.rpm is not installed

There is a very simple fix for this, as documented here and in other places:

rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY*

I don’t know how the system got into this state. It is a lab machine that gets used for many different experiments. I was happy to have found a simple fix.

What if ansible_default_ipv4 is Empty?

A colleague was attempting to use ansible to install Kubernetes, but he hit an error that confused him:

TASK [etcd : Write etcd config file] *******************************************

task path: /root/k8s-20160803-vishpat-contrib-git/contrib/ansible/roles/etcd/tasks/main.yml:23

fatal: [k8s-master.vnslab.net]: FAILED! => {"changed": false, "failed": true,
 "invocation": {"module_args": {"dest": "/etc/etcd/etcd.conf", "src": "etcd.conf.j2"},
 "module_name": "template"}, "msg": "AnsibleUndefinedVariable:
 {{ etcd_peer_url_scheme }}://{{ etcd_machine_address }}:{{ etcd_peer_port }}:
 {{ hostvars[inventory_hostname]['ansible_' + etcd_interface].ipv4.address }}:
 {{ ansible_default_ipv4.interface }}: 'dict object' has no attribute 'interface'"}

I asked him for a copy of the setup module (gather facts) for the host in question:

ansible -i 'your_host_name,' -m setup

This portion of the output jumped out at me:

<snip>
       },
        "ansible_default_ipv4": {},
        "ansible_default_ipv6": {},
        "ansible_devices": {
</snip>

ansible_default_ipv4 was empty. This was the root cause of the problem. When ansible tries to deploy the etcd template from roles/etcd/templates/etcd.conf.j2 it hits the following lines and attempts to substitute values for the variables:

<snip>
{% for host in groups[etcd_peers_group] -%}
  {{ hostvars[host]['ansible_hostname'] }}={{ etcd_peer_url_scheme }}:
    //{{ hostvars[host]['ansible_' + etcd_interface].ipv4.address }}:
    {{ etcd_peer_port }}
  {%- if not loop.last -%},{%- endif -%}
{%- endfor -%}
</snip>

And the definition of etcd_interface depends on ansible_default_ipv4 being populated. From roles/etcd/defaults/main.yaml: 

<snip>
# Interface on which etcd listens.
# Useful on systems when default interface is not connected to other machines,
# for example as in Vagrant+VirtualBox configuration.
# Note that this variable can't be set in per-host manner with current implementation.
etcd_interface: "{{ ansible_default_ipv4.interface }}"
</snip>

The result: When ansible tries to deploy the etcd.config template, it discovers that ansible_default_ipv4.interface doesn’t exist. It throws up its hands.

The fix: Setup a default route on the host under consideration. Instructions can be found here:

http://linux-ip.net/html/basic-changing.html#basic-changing-default

Once the change to to the host was made, ansible_default_ipv4.interface was populated! Problem solved!

Configuring Go CD for passwordless ssh clone from GitHub

I recently installed Go CD to do some CI/CD proof of concept work. (https://www.go.cd/) Go CD is a continuous delivery server based on pipelines of work to be accomplished. It integrates with several CRM systems, including GitHub.

After the install, I was prompted by the Go CD GUI to create my first pipeline. Each pipeline has 3 main parts:

  • Basic Settings
  • Materials
  • Stage/Job

Basic settings are simple: Name of the pipeline and the pipeline group it belongs to. It is in the setting up of the Materials that I ran into trouble.

The Materials page requires 3 pieces of information:

  • Material Type (GitHub)
  • URL (SSH clone URL)
  • Branch (master by default)

When I clicked “Check Connection”, I received the following error:

--- ERROR ---
STDERR: Permission denied (publickey).
STDERR: fatal: Could not read from remote repository.
STDERR: 
STDERR: Please make sure you have the correct access rights
STDERR: and the repository exists.
---

The problem is that Go CD Server runs as the “go” user on my Go CD master and slaves. I needed to add the “go” user’s public ssh key for each server (master and all slaves) to GitHub as a Deploy Key.

On each server (master and slaves), I did the following:

  • Login to the server using ssh
  • Change to the “go” user via ‘sudo su go’
  • Create the “go” user’s public key via ‘ssh-keygen’. I went with default path and file name (‘/var/go/.ssh/id_rsa’) and gave no passphrase.
  • Copied the contents of ‘/var/go/.ssh/id_rsa.pub’ to GitHub as a new Deploy Key.

On each server (master and slaves), I tested the connection by executing a command-line git clone. In each case, I was prompted to permanently add the GitHub server to my list of known hosts.

Once this was complete, “Check Connection” passed with “OK”.

 

launchd on Mac, part 2

In a previous post I wrote about my positive experience with a tool that automated the creation of launchd plist files for Mac, http://launched.zerowidth.com/. After several weeks of running, I found myself in a position of not knowing if the job was running as scheduled. A bit more searching yielded two interesting things.

First, I found that the plist syntax supports two keywords that allow me to capture the output of my script in files:

 <key>StandardErrorPath</key>
 <string>/Users/myname/backup.stderr.log</string>
 <key>StandardOutPath</key>
 <string>/Users/myname/backup.stdout.log</string>

I then changed the program arguments from this:

 <key>ProgramArguments</key>
 <array>
     <string>sh</string>
     <string>-c</string>
     <string>rsync -av -e ssh ~/Documents user@10.10.10.10:/home/user/backup/</string>
 </array>

to this:

 <key>ProgramArguments</key>
 <array>
     <string>sh</string>
     <string>-c</string>
     <string>date;rsync -av -e ssh ~/Documents user@10.10.10.10:/home/user/backup/</string>
 </array>

Now the stdout log file contains a timestamp for every time the task is run. I can simply open the file and scroll to the end when I feel uncertain about whether the task ran. 🙂

Second, I stumbled upon an excellent primer on launchd and the launchctl command that exercises it: http://nathangrigg.net/2012/07/schedule-jobs-using-launchd/

 

Writing a WebApp with Go

The standard packages available in Go for creating a WebApp are net/http and html/template. There is an excellent step-by-step walk-through of creating a simple wiki using Go located here:

https://golang.org/doc/articles/wiki/

The golang folks have done us a great service. The wiki builds from a bare-bones web server to a primitive wiki, showing us how to effectively use redirection, templates, and a number of other concepts.

My familiarity with Go has increased as a result of this tutorial. Bravo!

A Launchd.plist generator for passwordless rsync (and anything else!)

I was trying to figure out how to configure rsync to run on a schedule on my Mac. My goal was to backup my laptop’s Documents directory to an Ubuntu instance running in the lab. (My employer doesn’t support Macs very well. They have automatic backups for Windows laptops. Mac users are left to their own best effort.)

Rsync is a great option. When used in archive mode (-a) it conserves bandwidth by only sending changes over the wire. Sweet. The command I use is:

rsync -av -e ssh ~/Documents/ user@10.10.10.10:/home/user/backup/

Note the use of the ‘-e’ option to tell rsync to use ssh. I have copied my ssh public key to /home/user/.ssh/authorized_keys on the destination server. This allows rsync to run without requiring a password.

As I read through the options for scheduling on the Mac, launchd seemed to be the best choice for me. Configuration of launchd is both simple and complex. Simple because the creation of a plist file in ~/Library/LaunchAgents is all you need. Complex because the syntax of the plist file is powerful and potentially tedious. I didn’t find any “launchd for dummies” sites. Most sites I found repeated the same example. I wanted more.

Then I found http://launched.zerowidth.com/. This *very* useful site presents a form to fill out, then produces your plist file, and finally provides instructions for copying the plist file to your system and configuring launchd to use it immediately. The form takes the guesswork out of the scheduling portion of the configuration. Double sweet.

Here’s a screenshot of the Basics for running my backup job at noon on Monday, Wednesday and Friday each week:

Screen Shot 2016-05-23 at 11.34.44 AM

The screen, above, produces the following plist file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>Label</key>
	<string>com.zerowidth.launched.backup</string>
	<key>ProgramArguments</key>
	<array>
		<string>sh</string>
		<string>-c</string>
		<string>rsync -av -e ssh ~/Documents/ user@10.10.10.10:/home/user/backup/</string>
	</array>
	<key>StartCalendarInterval</key>
	<array>
		<dict>
			<key>Hour</key>
			<integer>12</integer>
			<key>Minute</key>
			<integer>0</integer>
			<key>Weekday</key>
			<integer>1</integer>
		</dict>
		<dict>
			<key>Hour</key>
			<integer>12</integer>
			<key>Minute</key>
			<integer>0</integer>
			<key>Weekday</key>
			<integer>3</integer>
		</dict>
		<dict>
			<key>Hour</key>
			<integer>12</integer>
			<key>Minute</key>
			<integer>0</integer>
			<key>Weekday</key>
			<integer>5</integer>
		</dict>
	</array>
</dict>
</plist>

I found that looking at the plist output was very helpful in understanding the syntax.

 

A Great Ansible First Tutorial

I have been using Ansible playbooks to deploy OpenStack. Using playbooks created by others is handy, but it doesn’t give one a sense of the power and flexibility of the tool. When I read the yml files themselves, it wasn’t always clear to me what was going on.

I searched online for a tutorial. I read through a few. As is typical of online technology tutorials, the authors frequently erred on the extremes of providing so much detail that we miss the bigger picture *or* they are written as if the author assumes that we know much that goes without being said. Sigh.

Then I found Michael Booth’s “one line and no files” introduction to Ansible:

http://www.mechanicalfish.net/start-learning-ansible-with-one-line-and-no-files/

It was written at the level of detail that let me have a few “so that’s how it works!” moments with less than 5 minutes of reading.

Booth’s post isn’t designed to tell us everything we need to know, but it does a great job of giving us an understanding of a few fundamentals. For me, it was enough to give me an understanding that Ansible is something I want to learn more about.