Thursday, October 30, 2014

Putting an iTunes Library on a NAS, Reliably

I have tried for years to get my iTunes library to reside, reliably, on my home NAS.  After all the NAS offers a ton of storage, resilience with RAID, and allows me to run lean on an SSD in my Mac Mini.  But for years I've run into issues where the network burps, or the NAS reboots, or coronal mass ejections cause my Mini to be disconnected from the NAS.  OK, not so much the last one, but when the Mini is disconnected from the NAS, I get the inevitable sequence of events.  First I get the little "Volume Disconnected" popup on my Mini.  Then something causes iTunes to realize that the network share with the library no longer exists.  Then iTunes silently creates a new iTunes library in my home folder, under Music.  Then I have two libraries to reconcile when I finally figure it all out -- usually when 95% of my library goes missing.

So naturally I think, "There's got to be a better way.  Microsoft figured out long ago in Windows how to automatically reconnect network shares if the NAS is disconnected from the server or workstation, why not Apple?"  Well, it turns out Apple has this figured out as well.  They just don't provide the user-friendly checkbox to enable the capability.

I found a couple of very useful resources on my latest search for an answer (thanks Google).  The first was this white paper that Apple released in 2009 that details how to use autofs.  For those that want to understand the full capability, it's the place to start.  For those that want the Cliff's Notes version, there's a great post on Superuser.com that explains how to get the solution up and running quickly.  I've read both, and I chose the indirect mapping method described in the post for setting up my auto-mounted share.  And so far, so good.  I purposely held off on a software upgrade on the NAS until after I made the auto-mount change to see how resilient the solution is, and it works great.  Be sure to tune down the AUTOMOUNT_TIMEOUT value in /etc/autofs.conf.  The default value is 3600 seconds.  I've tuned it down to 300 (5 minutes), and all seems well.

One side note: when you change from traditional mounting of the share via Finder to mounting via autofs, the mount point for the share changes as well.  While Finder will automatically mount the share under the /Volumes directory, autofs pretty much lets you mount anywhere.  Best practice (Greg practice?) is to mount the NAS shares under the /Network directory.  I mount mine as /Network/NAS_Name/share_name/ just in case I ever run multiple NAS'es in my house.  The good news is that it seems applications like iTunes and Carbon Copy Cloner identify the share by its network name (e.g., afp://user@NAS/share_name/) rather than the mount point (e.g., /Volumes/share_name/).  So when I changed from Finder-based mounting to autofs, I didn't have to tell any of the applications about the change in directory structure.  They simply picked up the /Network/NAS_Name/share_name/ change automagically.

One last note: you will lose the little drive icons that Finder puts on your desktop when you mount network shares with it.  Not a big deal, but if it bothers you, you can always just create aliases on the desktop instead.

Well, happy auto-mounting!

Update:  November 25, 2015
So while all seemed well when I originally wrote this blog, I've noticed on several occasions where iTunes opens before the NAS volume is mounted.  When this happens, iTunes silently reverts the location of my iTunes library to my local SSD.  So now rather than launch iTunes directly, I wrote this handy Automator script that will test for the existence of a file on the volume, and only launch iTunes if the file exists.  The beauty is that testing for the file will automatically mount the volume if it can be mounted:

if [ -f /Network/NAS/media/00-media-is-alive ]; then
    open /Applications/iTunes.app
else
    osascript -e 'display notification "Folder media is missing. Failed to start iTunes." with title "Automator"'
fi


Then just run the command "touch /Network/NAS/media/00-media-is-alive" from a terminal window and you're good to go.  Instead of launching iTunes directly, run the script instead.  If the script can verify that the file 00-media-is-alive exists, meaning that the volume is mounted, it will start up iTunes.  If not, then you will get a message in Notification Center telling you that iTunes failed to start because the volume with the iTunes library couldn't be mounted.

Friday, May 30, 2014

Junos Automation, Git, and MyCloud

Recently I've begun playing with tools for performing off-box automation with Junos, mostly thanks to an excellent class taught by Juniper's own Jeremy Schulman.  As part of that class, I created a personal github account where I can develop and share some of the automation tools I need (or might need) to make my life a bit easier.  (Don't look for too much there now, however.  I still have a day job, and this constitutes some after-hours learning and R&D.)

Having come from an organization that suffered mightily trying to develop a methodology for deploying standard configurations to a mishmash of devices in order to consistently and correctly provision end-to-end services, I immediately latched on to the idea of using Ansible to provide that structure, and the early results have been promising.  My "Hello World" in this case was a template and playbook that builds the [edit system] container of the Junos config file.  I've used it now to standardize that container across all of my lab devices and home network infrastructure as well.

As part of learning how to use Ansible, YAML, and Python to automate repetitive tasks, I also started playing with Github, and git more generally, as a revision control tool.  And that got me thinking -- not only can git provide revision control for scripting and software projects, it could do the same for Junos config files!  The only problem was that I didn't want copies of my config files showing up on Github, so I needed a git server that was totally under my control, and preferably hosted in-house (in my house), for this purpose.  I looked at all the usual places (Ubuntu, Centos, etc.) at first, but then found that git was available for the Western Digital MyCloud EX4 that I use as my home file server!  So very cool, git running on a box that already has a very nice RAID5 array spinning inside, and that regularly gets backed up to an external drive.  That sounded like an excellent solution for running revision control on my config files.  Just one problem -- no documentation, and I barely know what I'm doing with git.  There's a lot to it that I haven't learned/don't understand, but fortunately Google is my friend.  So here is how I turned my MyCloud into a git server...

Step #1:  Setting up Git
This process is relatively simple.  Start by installing the app from the MyCloud admin console.  Navigate to the Apps container and click the little + icon at the bottom of the left column.  Now check the git app and click Install.  That's all to configure from the admin console, at least for now.

Now you need to SSH to the MyCloud.  If you haven't already set up SSH access to the device, refer to this article for how to set it up.  Next you will need to setup a user for git.  You can just re-use one of your share accounts, but it's probably better to have a designated git user.  You can do this as follows:

~ # adduser git
adduser: /usr/share/ftp: File exists
Changing password for git
Enter the new password (minimum of 5, maximum of 8 characters)
Please use a combination of upper and lower case letters and numbers.
Enter new password: 
Re-enter new password: 
Password changed.

While you're at it, modify the contents of /etc/ssh/sshd_config to allow the git user to login via ssh.  This is kind of important because you can't just "su git" with busybox on the MyCloud.  Find the line in sshd_config that reads "AllowUsers root sshd" and change it to read "AllowUsers root sshd git".  Now restart ssh:

~ # ps ax | grep ssh
 1934 root      7448 S    sshd: sshd@ttyp0
10031 root      2716 S    grep ssh
21917 root      4836 S    /usr/sbin/sshd -f /etc/ssh/sshd_config
~ #
~ # kill -HUP 21917
(Replace 21917 with the PID your system reports.)

Before logging out, create a working directory for git.  I like /usr/share/ftp/git since it's one level below the default home directory for all regular users on the MyCloud:

~ # mkdir /usr/share/ftp/git
~ # chgrp git:share /usr/share/ftp/git

Now logout and log back in as the git user.  It's much safer that way.  Once you login, you should get dropped into the /usr/share/ftp folder.  Now it's time to build a bare repo:

~ $ cd git
~/git $ git init --bare myProject.git

That's all there is to setting up the repo.  Now step away from the shell and move back to the WebUI for a few minutes...

Step #2: Set up a Share for Hosting Code
This isn't completely necessary, but it's nice to have a place where I can go to pull the latest config files (or any project files) without needing access to my local git repository.  Start by navigating to the Shares container and clicking the + icon in the lower left again.  Give your new share a name (git) and a nice description.  Set the Public switch to the OFF position and set everyone's access to read-only.

OK, time to go back to the shell.  Create a project folder in the share you just created:

~ $ mkdir /shares/git/myProject

Now it's on to the last step.

Step #3: Automate the Content Refresh
Here comes the fun.  Navigate down to the hooks/ folder in your git repository and create a file named post-receive there as follows:

~ $ cd git/myProject.git/hooks
~/git/myProject.git/hooks $ cat post-receive
#!/bin/sh
GIT_WORK_TREE= git checkout -f
^D

In this example, is /shares/git/myProject.  One last thing, make the file you just created executable by git:

~/git/myProject.git/hooks $ chmod 744 post-receive

And there you have it.  Your MyCloud is now a git server with automatic checkout of the latest files to a read-only share!

Option: Pushing files to a full-fledged clone of your repo
Setting up a complete clone of your repo on the MyCloud is pretty straight-forward as well.  First, rather than simply creating a project folder under your git share, you can clone the repo:

~ $ cd /shares/git
/mnt/HD/HD_a2/git $ git clone /usr/share/ftp/git/myProject.git

Next, you need a different post-receive file.  Replace the steps in Step #3 with this:

~ $ cd git/myProject.git/hooks
~/git/myProject.git/hooks $ cat post-receive
#!/bin/sh
cd /shares/git/myProject
git pull myProject.git
^D

Boom!  Now you have a fully functional copy of your repo, automatically synchronized when you commit.