Friday, May 30, 2014

Junos Automation, Git, and MyCloud

Recently I've begun playing with tools for performing off-box automation with Junos, mostly thanks to an excellent class taught by Juniper's own Jeremy Schulman.  As part of that class, I created a personal github account where I can develop and share some of the automation tools I need (or might need) to make my life a bit easier.  (Don't look for too much there now, however.  I still have a day job, and this constitutes some after-hours learning and R&D.)

Having come from an organization that suffered mightily trying to develop a methodology for deploying standard configurations to a mishmash of devices in order to consistently and correctly provision end-to-end services, I immediately latched on to the idea of using Ansible to provide that structure, and the early results have been promising.  My "Hello World" in this case was a template and playbook that builds the [edit system] container of the Junos config file.  I've used it now to standardize that container across all of my lab devices and home network infrastructure as well.

As part of learning how to use Ansible, YAML, and Python to automate repetitive tasks, I also started playing with Github, and git more generally, as a revision control tool.  And that got me thinking -- not only can git provide revision control for scripting and software projects, it could do the same for Junos config files!  The only problem was that I didn't want copies of my config files showing up on Github, so I needed a git server that was totally under my control, and preferably hosted in-house (in my house), for this purpose.  I looked at all the usual places (Ubuntu, Centos, etc.) at first, but then found that git was available for the Western Digital MyCloud EX4 that I use as my home file server!  So very cool, git running on a box that already has a very nice RAID5 array spinning inside, and that regularly gets backed up to an external drive.  That sounded like an excellent solution for running revision control on my config files.  Just one problem -- no documentation, and I barely know what I'm doing with git.  There's a lot to it that I haven't learned/don't understand, but fortunately Google is my friend.  So here is how I turned my MyCloud into a git server...

Step #1:  Setting up Git
This process is relatively simple.  Start by installing the app from the MyCloud admin console.  Navigate to the Apps container and click the little + icon at the bottom of the left column.  Now check the git app and click Install.  That's all to configure from the admin console, at least for now.

Now you need to SSH to the MyCloud.  If you haven't already set up SSH access to the device, refer to this article for how to set it up.  Next you will need to setup a user for git.  You can just re-use one of your share accounts, but it's probably better to have a designated git user.  You can do this as follows:

~ # adduser git
adduser: /usr/share/ftp: File exists
Changing password for git
Enter the new password (minimum of 5, maximum of 8 characters)
Please use a combination of upper and lower case letters and numbers.
Enter new password: 
Re-enter new password: 
Password changed.

While you're at it, modify the contents of /etc/ssh/sshd_config to allow the git user to login via ssh.  This is kind of important because you can't just "su git" with busybox on the MyCloud.  Find the line in sshd_config that reads "AllowUsers root sshd" and change it to read "AllowUsers root sshd git".  Now restart ssh:

~ # ps ax | grep ssh
 1934 root      7448 S    sshd: sshd@ttyp0
10031 root      2716 S    grep ssh
21917 root      4836 S    /usr/sbin/sshd -f /etc/ssh/sshd_config
~ #
~ # kill -HUP 21917
(Replace 21917 with the PID your system reports.)

Before logging out, create a working directory for git.  I like /usr/share/ftp/git since it's one level below the default home directory for all regular users on the MyCloud:

~ # mkdir /usr/share/ftp/git
~ # chgrp git:share /usr/share/ftp/git

Now logout and log back in as the git user.  It's much safer that way.  Once you login, you should get dropped into the /usr/share/ftp folder.  Now it's time to build a bare repo:

~ $ cd git
~/git $ git init --bare myProject.git

That's all there is to setting up the repo.  Now step away from the shell and move back to the WebUI for a few minutes...

Step #2: Set up a Share for Hosting Code
This isn't completely necessary, but it's nice to have a place where I can go to pull the latest config files (or any project files) without needing access to my local git repository.  Start by navigating to the Shares container and clicking the + icon in the lower left again.  Give your new share a name (git) and a nice description.  Set the Public switch to the OFF position and set everyone's access to read-only.

OK, time to go back to the shell.  Create a project folder in the share you just created:

~ $ mkdir /shares/git/myProject

Now it's on to the last step.

Step #3: Automate the Content Refresh
Here comes the fun.  Navigate down to the hooks/ folder in your git repository and create a file named post-receive there as follows:

~ $ cd git/myProject.git/hooks
~/git/myProject.git/hooks $ cat post-receive
#!/bin/sh
GIT_WORK_TREE= git checkout -f
^D

In this example, is /shares/git/myProject.  One last thing, make the file you just created executable by git:

~/git/myProject.git/hooks $ chmod 744 post-receive

And there you have it.  Your MyCloud is now a git server with automatic checkout of the latest files to a read-only share!

Option: Pushing files to a full-fledged clone of your repo
Setting up a complete clone of your repo on the MyCloud is pretty straight-forward as well.  First, rather than simply creating a project folder under your git share, you can clone the repo:

~ $ cd /shares/git
/mnt/HD/HD_a2/git $ git clone /usr/share/ftp/git/myProject.git

Next, you need a different post-receive file.  Replace the steps in Step #3 with this:

~ $ cd git/myProject.git/hooks
~/git/myProject.git/hooks $ cat post-receive
#!/bin/sh
cd /shares/git/myProject
git pull myProject.git
^D

Boom!  Now you have a fully functional copy of your repo, automatically synchronized when you commit.