We wrestled with the decision to add it for some time, as we place a very, very high value on the simplicity of our systems. We have no intention of turning rsync.net into a development platform, running a single additional network service, or opening a single additional TCP port.
At the same time, there are a number of very straightforward synchronization and archival functions inherent to subversion and git that lend themselves very well to our offsite filesystem.
In addition, my own interest in DVCS in general, and git in particular as a tool for organizing plain old text files and unix file trees, added new urgency to the implementation.
That being said, we will not be adding any further such systems. There will not be cvs or Mercurial support in the future (although I am aware of some ways to run Mercurial over a git, or subversion "transport" and users are free to do so).
Some details...
Both git and svn can now be run without restrictions on the server side. Previously, we only allowed you to access svnserve through a svn+ssh:// style URL that you would pass to your own applications (like Tortoise, or the command line 'svn' tool). It was impossible, then, to create new repositories or sync from some other repository or view logs, etc. You are now able to do all of these things.
First, let's look at subversion.
We'll start by creating a remote repository in an rsync.net account:
ssh [email protected] "svnadmin create svn_repository"
The svn command is now supported and enables a vast set of functionality which previously wasn't available. For example, if we wanted to keep an up-to-date working copy of one of our projects for which a repository was remotely available (we'll use http://example.com/svn/repository as our remote URL), we could issue the following commands:
ssh [email protected] "svn checkout http://example.com/svn/repository my_checkout" ... list of files checkeChecked out revision 42.
If you'd like to update that checkout later, you can issue a command like:
ssh [email protected] "svn update my_checkout"
The svnsync command is also available. svnsync allows synchronization of repositories. This is useful to keep a full backup of an existing repository. Again, we'll use the URL of http://example.com/svn/repository as the repository for which we want to create a backup. First, we'll create a repository:
ssh [email protected] "svnadmin create synced_repository"
svnsync requires that we initialize the repository prior to doing any syncing. Part of this initialization process requires that we change repository attributes that are normally restricted, so we need to allow these properties to be changed by creating a hook script that exits with a zero return code. In order to do this, we'll create a symlink to the "echo" program:
ssh [email protected] "ln -s /bin/echo synced_repository/hooks/pre-revprop- change"
With that hook script in place, we can now initialize our repository for syncing. Before we do so, we need to know the exact path to our checkout, so let's discover the path to our home directory:
$ ssh [email protected] "pwd" /data2/home/1234
$ ssh [email protected] "svnsync initialize file:///data2/home/1234/synced_repository http://example.com/svn/repository" Copied properties for revision 0 (svn:sync-* properties skipped).
Now, we can issue the following command as often as needed in order to keep the repository up-to-date:
$ ssh [email protected] "svnsync sync file:///data2/home/1234/synced_repository" Committed revision 1. Copied properties for revision 1. ... Committed revision 42. Copied properties for revision 42.
We can also dump a repository to a file by using the 'svnadmin dump' command. Similarly, the database can be restored by using the 'svnadmin load' command. These two commands work in harmony to keep backups of subversion databases, or specific revisions within a database.
To make a single-file backup to my local machine of a repository existing on rsync.net, we can do the following:
$ ssh [email protected] "svnadmin dump existing_repository" > repository_backup.dump 2>repository_backup_errors.txt
The above command creates two files on the local machine, a repository_backup.dump file and a repository_backup_errors.txt file. The former is a single-file backup of the subversion repository and the latter is a list of notifications and errors that might be important.
Finally, the svn client can provide information about a checkout by using the "svn info" command. You can use this information to determine the source repository:
$ ssh [email protected] "svn info existing_repository" Path: . URL: http://example.com/directory Repository Root: http://example.com/directory Repository UUID: dc7efa32-411c-0410-9537-da5bd19367fc Revision: 42 Node Kind: directory Schedule: normal Last Changed Author: admin Last Changed Rev: 42 Last Changed Date: 2010-02-02 12:56:03 -0800 (Tue, 02 Feb 2010)
The svnlook command allows inspection of a repository itself. For example, to inspect the subversion repository and find out its UUID:
$ ssh [email protected] "svnlook uuid svn_repository" dc7efa32-411c-0410-9537-da5bd19367fc
Let's next examine the new git support.
All standard git operations that do not require external programs are supported when the repository is hosted by rsync.net. (See note about client support at the end of this blog post)
To create a git repository on rsync.net we can issue the following command:
$ ssh [email protected] "git init --bare git_repo.git" Initialized empty Git repository in /data2/home/1234/git_repo.git/
Note that the full path to the repository will be required for some operations.
Once a repository has been created, we would want to clone it so we have a local copy on which we could make changes. We can do so with the git clone command, but in order to do so we need the absolute path that was displayed when we initialized the repository:
$ git clone ssh://[email protected]/data2/home/1234/git_repo.git Initialized empty Git repository in /home/kibab/git_repo/.git/
Once you've made changes to your local repository, changes can be pushed back to rsync.net using "git push". By default, any changes we make on our local filesystem will be applied to the repository will be applied to the "master" branch. So, when we push changes up to rsync.net, we'll need to specify which branch should be pushed. When the "git clone" command was issued, git recorded the original repository in the .git/config file.
(NOTE: if you happen to have a hosts file alias for rsync.net, you might need to edit $REPO/.git/config so that if git attempts to bypass the hosts file lookup rsync.net may be correctly resolved.) Since git already knows where the origin (i.e. original) repository is, we can simply issue a push on the local filesystem:
$ git push origin master Counting objects: 3, done. Delta compression using up to 4 threads. Compressing objects: 100% (2/2), done. Writing objects: 100% (3/3), 3.33 MiB | 72 KiB/s, done. Total 3 (delta 0), reused 0 (delta 0) To ssh://[email protected]/data2 * [new branch] master -> master
After pushing the master branch, we'll no longer need to specify the branch we're working with as it will have contents and be selected by default unless otherwise changed.
To pull updates from your git repository, in the event that a different machine pushes some changes up to the git repository, use 'git pull':
$ git pull remote: Counting objects: 4, done. remote: Compressing objects: 100% (3/3), done. remote: Total 3 (delta 0), reused 0 (delta 0) Unpacking objects: 100% (3/3), done. From ssh://usw-s003.rsync.net/data2/home/1234/git_repo f4d263b..445595a master -> origin/master Updating f4d263b..445595a Fast-forward TestableJava.pdf | Bin 0 -> 86224 bytes 1 files changed, 0 insertions(+), 0 create mode 100644 TestableJava.pdf
Finally, we can make changes with branching. For this example, let's assume that we've been baking up a settings directory and we're about to embark on a system upgrade that may drastically change all of your settings files. We want to be able to upgrade the settings directory without losing any of the prior work and while retaining the ability to go back to the older settings, if necessary. To do this we'll need to create a new branch:
$ git branch upgrade
We can then see the branch using "git branch" without any parameters:
$ git branch * master upgrade
The asterisk (*) indicates that we're still using the master branch, so let's switch to the upgrade branch:
$ git checkout upgrade Switched to branch 'upgrade'
Now, after performing the upgrade and committing our changes, we can push the changes to the server without losing any of our prior work.
$ git push origin upgrade Counting objects: 4, done. Delta compression using up to 4 threads. Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 1.23 KiB, done. Total 3 (delta 0), reused 0 (delta 0) To ssh://[email protected]/data2/home/1234/git_repo.git * [new branch] upgrade -> upgrade
I can now switch between the two different versions using "git checkout" and use "git diff" to see the differences between the revisions. To revert back to our pre-upgrade configuration, we use "git checkout" one more time:
$ git checkout master Switched to branch 'master'
A special note regarding git client functionality:
git has been configured to be a fully functional server hosted on rsync.net. Although certain operations may work, not all programs have been installed to make it a fully functional client.
A further note regarding hook scripts:
There is no hook script support for either subversion or git. However, as we saw in the above subversion example, where svnsync required the pre-revprop-change hook script, you can symlink /bin/echo to the hook script you need, in your own rsync.net filesystem, to satisfy this rare requirement. It simply returns a zero and satisfies svnsync.
As always, please contact [email protected] if you have any questions.