Version Control and Online Music Repository Integration in The Last Homely Hobbit

Beginning with the 3.0 release, The Last Homely Hobbit now offers file version control and the ability to integrate with a cloud-based repository for the storing and sharing of ABC music. This offers a number of advantages to LotRO musicians:


Git... Not Just For Source Code

Git is the underlying software which powers The Last Homely Hobbit's file version control and online repository integration. You have probably heard of Git already: Maestro, perhaps the most widely used tool in the music community, is hosted on github, which is a Git-based repository site. Although Git is most commonly used to manage the source code behind various software programs, it is a version control system that can be used for any kind of computer file. In this case, we are using it for our ABC files - the source code behind the music we play in-game.


File Version Control

Let's take a quick look at how this works. File version control is available to you whether or not you choose to use an online repository, so let's skip the online part for now to get right to the demonstration. To begin, access the Version Control / Repository Integration option on the Settings menu:





Check the box to enable version control. To use The Last Homely Hobbit for file version control without using an online repository, you only need to fill out the first box, Managed Music Folder Name:



Once file version control has been turned on, The Last Homely Hobbit will create a new folder within your main LotRO music folder. Music files that you place within this folder will be included in the file versioning process, and music files you keep outside of this folder (in any of the other folders you have created within your LotRO music folder) will not.

The Managed Music Folder Name option is here to allow you to select the name that The Last Homely Hobbit will give to the managed music folder that it creates within your LotRO music folder. In the screenshot above, I chose the name ManangedLocal.

There are a couple of rules to keep in mind. The Managed Music Folder Name textbox accepts only upper and lower case letters. Also, you cannot choose an existing folder as your managed music folder - The Last Homely Hobbit needs to create it as part of initializing file version control.


Committing Changes

Now that you have enabled file version control, you'll notice a couple of things right off. To begin, you'll find that The Last Homely Hobbit created the managed music folder with the name you selected, and added a readme.txt file to that folder explaining its purpose. You'll also see some new elements in the user interface on the righthand side of the main window:



The arrow in this picture is the means by which you commit changes to your file version history. Let's stop for a brief moment and talk about what that means.

When you are working on an ABC file, you may edit it many times in the course of fine-tuning it - adding another track, changing the volume, etc. You don't really want to forever preserve the state of your new song after each of these edits - it's a work in progress. But eventually, you'll reach a milestone - a point where you feel you have a finished music file. This is the point at which you should commit, which means you'll be taking a snapshot of anything you've added or changed since your last commit - a snapshot which will be retained indefinitely, even after you have made more changes to that same file.

Let's see how it works. For demonstration purposes, I added a folder named Personal Songs to my managed music folder, and then copied an ABC file into it. After noticing the new additions, The Last Homely Hobbit updated its music database, and the number next to the commit arrow changed:



This number is telling us that one change has been detected since our last commit. If I hover over the number, I'll see a little popup which confirms that our new file is waiting to be committed:



I'm ready to commit this, so I just click the little arrow to the left of the number, and presto: it's done. Now that I've committed my changes, the number goes back to reading zero - no new changes since last commit. Note that you do NOT need to commit changes one at a time - you can commit hundreds or even thousands of changes at once.


Editing Our New Song

Let's have a bit of fun with our new song now. If we look at the parts readout, we'll see that this song includes a horn part:



Let's change it to a pibgorn part instead. Now, I know what you're thinking - changing a horn part to a pibgorn part is an artistic crime that is punishable by death in most civilized nations. But we're committing this heinous act in order to see how easy it is to undo it later:



Crikey. I can't believe we just did that. Painful as it may be, let's click the little commit arrow again in order to memorialize this change.

It's been ten seconds and already I'm missing my lovely horn part. I want it back! So, I am going to highlight the song row in the main music view, and then right-click it:



There's a new option on the context menu here, one you won't find on songs that live outside of the managed music folder. View Version History... let's see what it does.



We are brought to a new window with three rows for this song listed. The top row is the Current Version row: it represents the song as it currently exists in our LotRO music folder. But there are two other rows as well, one for each of the commits that we did. Remember when we said that committing was like taking a snapshot? Here are the pictures.

You'll find that you can play and view the prior versions of the song using the little buttons next to them, all without disturbing the copy that's sitting in the LotRO music folder. But I want to disturb that copy! I want my horn part back!

In addition to the play and view buttons, the historical versions of the song have a little grey arrow next to them in the Revert column. All I need to do is click the Revert arrow next to the earliest snapshot - the version that has the horn. After warning me that it's about to overwrite the version in my LotRO music folder with this historical copy (and giving me the chance to change my mind), The Last Homely Hobbit helpfully spits this much more preferable copy back where it belongs in my LotRO music folder. The UI updates again to reflect this change:



The horn part is back! And notice that my pending commit number has gone back to one. File version control is a like a river - always moving forward. This latest change - reverting to the version with the horn - is just another revision in the overall lifetime of this song, another change to be committed. There's no hurry, since I already have this exact version in my history for this song, but the reversion to the horn part will be immortalized the next time that I commit a batch of changes.

Incidentally, remember to keep the Keep My Songbook Plugin Data Updated option turned on (it's under Settings > Manage Settings) - all of these changes will be automatically pushed to Songbook as soon as you make them!


Limitations

Take another look at the screenshot of the Song Version History above. Do you see the Repository Path column? It's a representation of the song's filename, combined with its folder location within the managed music folder. This folder/filename combination represents the only way to unqiuely identify a song across the many commits in your music repository's history. Consequently, it is the hook that The Last Homely Hobbit uses to find prior versions of any given song. When you say you'd like to see a song's version history, The Last Homely Hobbit looks through the repository history for any activity involving that Repository Path name.

If for any reason, after initially committing a file, you change the folder in which you store it, or you change its filename, you are creating a break in that song's history. The committed versions of that song prior to you moving/renaming it still exist, but The Last Homely Hobbit can no longer find them due to their Repository Path being different from what it is now. So, to ensure the smoothest possible experience with file versioning, it pays to give some thought to which folder you want to keep something in, and to adopt a naming scheme that is unlikely to change over time, prior to first committing a file.

So, that's file versioning. Let's move on to discuss online repository integration, which gives us all of the same file versioning benefits, plus a whole lot more.



Online Repository Integration - Getting Started

If you want to realize all of the benefits of a cloud-based music repository, you'll first need to create a Git repository to host all of your music. There are multiple available options you can choose - I prefer gitlab.com, as they seem to have the most robust features for hosting private repositories for free. Register for an account at your provider of choice, and then create a new repository. You'll want to seed your new repository with at least one file, if your Git provider of choice doesn't do this for you automatically (GitLab did not) - we need at least one commit in there to establish a baseline. Adding a simple readme.txt file will do.

Your new repository will have its own unique internet address. At GitLab, this looks something like the following:

https://gitlab.com/YourGitAccountName/YourGitRepoName.git

You will need this address, as well as the username and password you created for yourself at your Git provider, in order to complete the setup within The Last Homely Hobbit.


Playing Solo or With Friends?

If you will be using your repository solely for your own personal use, there is nothing more to be done at your Git provider's site - you just need to complete the setup within The Last Homely Hobbit, which we will cover in a just a bit. However, if you will be allowing others to access your music repository, there is a bit more to do.

Your fellow musicians and bandmates will need their own accounts at the same Git provider you used to create your repository. Once they have signed up for a username and password, you can then grant them access to your music repository (identifying them using the Git provider username they created for themselves). As you grant each new individual access to your repository, you can also assign them a role which will govern their level of access.

If you are granting access to someone solely so that they can download the latest updates from your online music library, you'll want to grant them read-only access. At GitLab, assigning someone the Reporter role accomplishes this. If you'd like for the new user to be able to add, change, and delete music files within the repository, they'll need permission to commit to the master branch, which is the main unit of files within a repository (and it is the only branch used by The Last Homely Hobbit). Within GitLab, assigning someone the Master role allows this level of access.


Setup Within The Last Homely Hobbit

And that's all the external setup that is required! All that is left is to configure The Last Homely Hobbit to make use of your new repository. To begin, access the Version Control / Repository Integration option on the Settings menu:




You will see a screen like this:





Click the Enable Online Music Repository Integration checkbox to display the other settings fields:



We need to fill out all of this information, so let's take them one by one.




Once online music repository integration has been turned on, The Last Homely Hobbit will create a new folder within your main LotRO music folder. This new folder will act as the home for your local copy of the music which is hosted in the online repository. Music files contained within this folder will be included in the synchronization process between your local PC and the online repository, and music files you keep outside of this folder (in any of the other folders you have created within your LotRO music folder) will not.

The Managed Music Folder Name option is here to allow you to select the name that The Last Homely Hobbit will give to the folder containing your copies of the music files hosted in the repository. Choose a short, descriptive name like cloud or online or managed - anything which will help you recognize this as the folder which integrates with the online repository.

There are a couple of rules to keep in mind. The Managed Music Folder Name textbox accepts only upper and lower case letters. Also, you cannot choose an existing folder as your managed music folder - The Last Homely Hobbit needs to create it as part of the initial repository synchronization process.


Remember earlier when you learned that your Git repository has its own unique internet address? You need to paste that address into this box.


Here you should type the username you created for yourself at your Git provider's site.


Here you should type the password you created for yourself at your Git provider's site.


How Repository Integration Works

Once you have finished configuring The Last Homely Hobbit with the information it needs to integrate with your online repository, you will see some new elements in the user interface on the righthand side of the main window:



Let's examine each element separately, and discuss their purpose.




The arrow in this picture is the mechanism by which you download all pending updates from the online music repository (if you ever forget which arrow does what, just hover over them for a tooltip explaining their purpose) - click it to download all pending changes. The number next to the arrow indicates how many changes exist in the online repository which you have yet to download. You can hover over the number to get a detailed listing of the changes (the first 15 or so changes will be included in the little popup). The zero that is pictured indicates that there are no changes to be downloaded, and the popup we see in response to hovering the mouse pointer over the zero confirms this.




As you have probably guessed, the other arrow is the mechanism by which you upload to the online repository any music files that you've added or changed within your local managed music folder.



Click this arrow to upload the changes and additions in your local managed music folder to the online music repository. Again, if you ever forget which arrow does what, just hover over them for a tooltip explaining their purpose. Remember that you must have sufficient privileges in the online repository in order to be able to upload changes to it.

The number next to the uploading arrow indicates how many changes exist in your local managed music folder which you have yet to upload to the online music repository. You can hover over this number to get a detailed listing of the changes (the first 15 or so changes will be included in the little popup). The number pictured indicates that there is one change to be uploaded (a file I dropped into my managed music folder for illustration purposes), and the popup we see in response to hovering the mouse pointer over the number confirms this by showing us more detailed information.



Lastly, clicking the little refresh icon tells The Last Homely Hobbit that you'd like to check to see whether the online music repository has any updates.



Once this check has taken place, the number of available changes next to the download arrow will update accordingly.


And that's all there is to it! Once you are set up, it is quick and easy to synchronize your managed music folder with the online repository. To begin, you can seed your new repository by dropping all of the music files you want to store there into your managed music folder. You can have as many subfolders as you require for organizational purposes. Once you are set, the number next to the upload arrow will have registered your changes - simply click the arrow to commit your files to the repository.

It is also easy to see when there are changes waiting to be downloaded, which makes publishing updates to your bandmembers simple. You don't even need to tell them when the changes are made - they can see it for themselves as soon as they're available! And don't forget to turn on The Last Homely Hobbit's Keep My Songbook Plugin Data Updated feature (it's under Settings > Manage Settings) - all of these changes and additions will be automatically pushed to Songbook as soon as you've downloaded them!


Merge Conflicts

Wait, what? Any section with a title like Merge Conflicts can't be about anything good.

If you are the only one with access to your online music repository, or are the only one allowed to upload updates to the online music repository, then this section does not apply to you. But if you are integrated with a shared online repository that has multiple contributors, then the possibility for file conflicts exists.

Here's how it works: Git keeps track of exactly what version of each music file you have whenever you sync up with the online music repository. If you then change a music file in some way and try to upload your changes to the online repository, Git checks to see whether the version in the repository matches what you had before you made your changes. If it does, then you are the only person to have changed that file, and you can upload your changes without issue. But what if the version that's now in the repository is not the same as what you had the last time you synchronized? That means someone else changed that same file too, and they already uploaded their changes!

When two people have made different modifications to the same file between synchronizations, a state of conflict exists (it is called a merge conflict because merging is Git-speak for synchronizing changes between your local managed music folder and the online repository). The Last Homely Hobbit must resolve this conflict somehow in order for the file sharing process to continue smoothly.

The Last Homely Hobbit resolves conflicts by picking a winner - one of the two conflicting files will be kept, and the other discarded. To keep things simple, The Last Homely Hobbit treats the online music repository as the ultimate source of truth in every conflict situation - its version of the file is always picked as the winner.

So what does this mean for that file you painstakingly updated? To resolve the merge conflict, The Last Homely Hobbit replaced it with the version from the online music repository. But it didn't throw your version away! Instead, it saved your copy in a special folder outside of the managed music folder (the stash folder). Whenever this happens, you will be informed that a merge conflict was detected, and you will see a message box that explains where to find your stashed version of the conflicting file. You can then review both versions of the file (the stashed version and the version in your managed music folder), and take whatever steps you deem necessary to integrate your changes with the changes in the file you downloaded from the repository.

Whew! That was a lot of blather about something which, in practice, probably won't happen to most people all that often. But it's important for you to be aware of the potential for file conflicts, and what The Last Homely Hobbit does to resolve them.


Repo-pocalypse Now

A shared repository with multiple active contributors can quickly create some convoluted file conflict situations. I've invested a lot of time in running through all sorts of simulations and test cases, but the possibility exists that you could encounter a conflict situation that The Last Homely Hobbit cannot seem to resolve on its own. If your local repository ever falls out of sync with the online repository in a way that The Last Homely Hobbit can't work out on its own, all is not lost. Here is what you can do to kick start things back into smooth operation:
  • Close out of The Last Homely Hobbit completely.

  • Grab a copy of any recently added or changed files - that you haven't yet been able to upload to the online repository - from your managed music folder, and temporarily stash them somewhere else (outside of the managed music folder).

  • Delete your managed music folder from the main LotRO music folder.

  • Start up The Last Homely Hobbit again. As it goes through its startup routine, it will notice that you are configured for online music repository integration and that you are missing your managed music folder. This will cause The Last Homely Hobbit to recreate your managed music folder and to clone your online music repository, thereby forcing you back into sync.

  • Reintegrate the updates you stashed in step two back into your library, and then upload the changes to the online repository.

Hopefully you'll never need to go to this extreme, but this tactic can be used to resolve the worst of the worst file conflict situations.