Last weekend was a PFZ meeting. And for this day a few interesting workshops were planned. I mainly was looking forward to the GIT (Git for Subversion Users) talk by Stefan Koopmanschap. And the Dependency Injection presentation by Berry Langerak I didn’t make it too the last one. But learned a lot while doing the GIT workshop with Stefan.

For the last few months i have wanted to take a look at git. But it just never really took the time to sit down and do it. So this opportunity was there at the right time. Although my laptop died halfway through the workshop. I still was able to make it through the whole session together with Joost van Veen. Thanks man. We had a great time. And it was eye opening. Stefan gave a good talk. Helped out where possible. And besides a small Github glitch. It was a successful day.

So successful that i decided to make the switch from Subversion to GIT. Below are some of the steps i took.

The main reasons for me to make the switch

• Distributed structure make it easy to work from any place. connected or not.
• Stashing
• Speed and cleaner folder / file structure
• Real tags

That’s all for now. but this list will grow. I am sure!

Setting up a GIT server

I decided to go for my own GIT server instead of Github. That’s just the way i like it. Full control.

$ aptitude install git-core

Now all we have to do is create a user for git operations. And set the correct permissions.

$ adduser git
$ mkdir /home/git/.ssh
$ chown -R git:git /home/git/.ssh
$ chmod 700 /home/git

If we want to create a new remote repository. We log in on the git server through SSH and setup a new repository like this.

$ ssh git@git.domain.tld
$ mkdir repository.git
$ cd repository.git
$ git –bare init

That’s all. we now have all we need to connect to git remotely and do some work. But now first we need to import the old Subversion repository. I have found two solid methods for getting the job done.

Import a SVN project into GIT with git-svn

First of install git-svn

$ sudo aptitude install git-svn

Before we start the importing. It’s good practice to create an author mapping file. So the SVN commiter names match the GIT ones. This is done by creating a simple text file like so.

$ touch authors.txt
$ echo “m0s = Thijs Lensselink ” > authors.txt

Importing the existing tags and branches is done by adding the –stdlayout switch

$ git svn clone –stdlayout -A authors.txt svn+ssh://user@svnhost/repository repository.git

The problem with this method is that tags and branches are not created properly. I decided to go for the second method down here.

Import a SVN project into GIT with svn2git

For this to work we first need ruby and some gems packages.

$ apt-get install git-svn ruby rubygems
$ gem install svn2git –source http://gemcutter.org

Create the local repository and import the files from SVN

$ mkdir repository.git
$ cd repository.git
$ /var/lib/gems/1.8/bin/svn2git svn+ssh://user@svnhost/repository –authors ../authors.txt

Show the current selected branch

$ git branch
* master

Show all branches. Notice the trunk branch

$ git branch -a
* master
trunk

List all tags

$ git tag -l
v1.0
v2.0

That seemed to work pretty good for almost all projects. Some projects didn’t have the main trunk in SVN. So to import these just add the –rootistrunk switch.

And while this info is still available in my brain at the moment. I will write down some of the workshop material for future reference.

Setup local Development environment

Create directory somewhere that will serve as the local repository. And run init to create the basic git project structure.

$ mkdir [project name]
$ cd [project name]
$ git init

Now we have a fully functional git repository. We can do anything with (add, commit, tag, branch, push, etc.) The first things to do when starting out is setup the user name and email address of the developer using this repository. This is good when looking though the GIT project logs later on. Also handy to set the core editor to the most convenient editor available.

$ git config –global user.name “Thijs Lensselink”
$ git config –global user.email “email@lenss.nl”
$ git config –global core.editor vi

The repository exists out of a origin and a master branch at this moment. The origin refers to the current repository location. And the master refers to the current repository state. Like HEAD in CVS and SVN. Let’s add a file for testing.

$ touch index.php
$ git add index.php (or add *)
$ git commit -m “Commit something”

Working on a local repository is nice. but we want to save it on a server of course. For this we need to set a repository location. We can do this in two ways. We can overwrite the origin or add a new location. Let’s try the first one.

$ git remote add origin git@git.domain.tld:repository.git

Setting up the remote git repository and push the changes

All changes to this repository are made locally so far. Let’s try and push them to the git server. The only problem here is. I haven’t found a way to create repositories remotely. So we have to log in to the server here and create the repository.

$ ssh git@git.domain.tld
$ mkdir repository.git
$ cd repository.git
$ git –bare init
$ exit

Back on the client side we can now do a push an see what happens.

$ git push origin master
Counting objects: 3, done.
Writing objects: 100% (3/3), 232 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To git@192.168.178.24:foo.git
* [new branch] master -> master

if you get this error:

error: src refspec master does not match any.

This means there is no master branch yet. This happens when the repository was created with the –bare switch. You have to commit a change first before you can push anything to the remote repository server.

That’s all. We now have a working GIT server and local repository. All that is left is to do some serious development work. On the other hand, before i forget. i will use this space to rehash Stefan’s workshop for future reference.

Tagging

Let start by looking at the way git creates tags. First of there are two types of tags lightweight and annotated. The lightweight version is nothing more then a pointer to a specific commit. It’s like a branch that doesn’t change. Annotated tags are a bit different then their lightweight version. And for sure different from the tags in Subversion. A tags in git contains the following data

• checksum
• tagger name
• email + date
• tag message
• GPG signed

Creating an annotated tag for version 1 looks something like this

$ git tag -a v1.0 -m “Creating a tag”

Dispaly a certain tag

$ git show v1.0

I haven’t had time to look into working with signing of tags. So maybe i will add an entry about that later on.

Branching

A repository can have multiple branches. And switching between branches is a since in git. Branching is much like
it is in Subversion.

Create a new branch from master called foo

$ git branch foo master

Display all branches available. The current branch is highlighted by the * sign

$ git branch
* master
foo

Changing branches in git is a piece of cake.

$ git checkout foo
Switched to branch ‘foo’

There is not much more to branching.

diff, merge and delete branches

When adding a new feature to an existing code base. It’s good practice to create a branch before you start editing. And after everything is done and the change is stable. The branch can be merged back into the master.

First let’s check if there is a difference between the newly created branch and the master.

$ git diff foo master
diff –git a/touch.foo b/touch.foo
deleted file mode 100644
index 257cc56..0000000
— a/touch.foo
+++ /dev/null
@@ -1 +0,0 @@
-foo

Some changes have been found so let’s merge them in the master branch

$ git checkout master
$ git merge foo master (or git pull foo master)
Already up-to-date with 12d2f9cebaf71a580b021a8eeddc38f267b69e53
Trying simple merge with 4d3dff7e0d2a26302a5884e763fd7a48f5ab4437
Merge made by octopus.
touch.foo | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 touch.foo

with git pull it is possible to import branches from other repositories

When done with the changes. And everything is pushed to the remote git server, we can remove the ‘foo’ branch

$ git branch -d foo
Deleted branch foo (was d9a8c76).

That’s all for branching right now.

Stashing

One of my personal favorite the stash command. Imagine working on a project. And somebody asks you to make a small change. In subversion i needed to do a new checkout and work on that. This can be quite slow and cumbersome. In git this is solved elegantly by using the stash command.

$ git stash
Saved working directory and index state “WIP on master: 648f347… foo”
(To restore them type “git stash apply”)
HEAD is now at 648f347 foo

This saves a copy of the working state and let’s you do changes on a fresh base. After your done get back to your work by doing

$ git stash pop (or apply)

With pop you push the stash back to the master. With apply the changes will first be applied to the master before restoring the work state.

Working with other repositories

Clone a repository from a remote GIT server

$ git clone git@git.domain.tld:repository.git ./project2

Clone a local GIT repository

$ git clone ./project ./project2

Now it could be possible a friend or a remote community member without write access made a few changes to his / her remote repository. And wants them on the global remote GIT server. This can be done by pulling the code.

First add the new repository location

$ git remote add friend git://githubaddress

And then fetch the remote changes

$ git fetch friend

This only fetches the changes. but does not apply them to the master. Apply them

$ git pull friend master

updating remote repository

$ git push friend master

That’s all i can remember right now.

I was kinda bored yesterday. And was poking around the SVN hooks scripts. When i noticed the post-commit script i was using was written in Python. Nothing wrong with that. I just wondered how hard it would be to do the same in PHP. So i came up with a little test. Which at this moment lacks error checking and just looks plain ugly. But it does the job. and i only needed a working PHP CLI executable to make it work.

First we set the email address and the location of svnlook.

#!/usr/bin/php
<?php
error_reporting(E_ALL | E_STRICT); 

$address = 'some email address';
$svnlook = '/usr/bin/svnlook';

Use the argv array to get to the commandline parameters and build up a SVN statement to get some basic info.

$repository = $_SERVER['argv'][1];
$revision = $_SERVER['argv'][2];

$cmdInfo = "${svnlook} info ${repository} -r ${revision}";

$output = array();
exec($cmdInfo, $output);

$author = $output[0];
$date = $output[1];
$comment = $output[3];

Now let’s get the changes made in this revisions.

$output = array();
$cmdChanges ="${svnlook} diff ${repository} -r ${revision}";
exec($cmdChanges, $output);

Once we have the diff contents we can loop through the lines and use some simple regex to highlight the changes. I’ve added a bit more code to add pre tags for a better visual effect.

$patch = '<pre class="codeBlock">';
$lineCount = count($output);
for ($x=0; $x<=$lineCount; $x++) {

    if ($x > 0 && strstr($output[$x], 'Modified:')) {
        $patch .= '</pre><pre class="codeBlock">';
    }

    $textColor = 'normal';
    if (preg_match('/^\+/', $output[$x])) {
        $textColor = 'positive';
    }
    else if (preg_match('/^-/', $output[$x])) {
        $textColor = 'negative';
    }

    $patch .= '<span class="' . $textColor . '">' . htmlentities($output[$x]) . "</span><br>";

    if ($x == $lineCount) {
        $patch .= "</pre>";
    }
}

Now we can build the body for the email that will be generated.

$body .= "Author : " . $author . "<br>";
$body .= "Date : " . $date . "<br>";
$body .= "Comment : " . $comment . "<br><br>";

$body .= $patch;

Add a MIME header so my mail client displays it properly. And send it of through the build in mail() function.

$headers  = 'MIME-Version: 1.0' . "\r\n";
$headers .= 'Content-type: text/html; charset=iso-8859-1' . "\r\n";

$bodyHTML = file_get_contents("/path/to/some/HTML/template");
$bodyHTML = str_replace('{EMAIL}', $body, $bodyHTML);

mail($address, 'SVN post-commit', $bodyHTML, $headers);

Yesterday my colleague was complaining he couldn’t do a normal check out of one of the projects in Subversion. At first i thought it was just something wrong with his local settings. Because i could do the check out just fine. Today when i was thinking about this problem. I quickly realized we are on two different platforms. He works on Windows while I’m on Linux.

And since i had to do some Photoshop work today. I took the chance to install Subversion on my Windows partition. When i did a checkout of the failing project. I ran into the same problems as my colleague did before. Two different errors were preventing me from making a complete checkout. The first error:

svn: In directory ‘path/to/some/images’
svn: Can’t copy ‘path/to/some/images/image.JPG.svn-base’ to ‘path/to/some/image.JPG.tmp’: No such file or directory

Is caused by multiple files with the same name. At  least to windows it looks like this. Because image.jpg and image.JPG are two different files on Linux. On windows however these two files are one and the same. This causes Subversion to choke. After manually deleting all duplicates from the repositorie it self the problem dissapeared.

The second error was something like:

svn: Working copy ‘/some/path/\ ‘ is missing or not locked

This one was actually pretty obvious. A file name with an illigal character. I wonder how it got in the repository. The folder name causeing the problems was a folder called ‘\ ‘. Which was empty. After removing this. I could do a complete checkout again.

This should be pretty damn straight forward. However i ran into some problems while setting this up. At my daily work spot we use SVN over HTTP so we can do NTLM authentication. This sucks in many ways. But most of all it’s slow. And i mean really slow. Last week i tried to retrieve the log information for a project in the repository. This took my IDE 30 minutes. At first i thought my IDE was just being lame on me. But it turns out it’s SVN over HTTP that’s the culprit.

So i started to do some testing over the standard svn:// protocol. And retrieving the logs on the same repository took about 2 seconds. Amazing. That’s enough reason to switch to the svn:// protocol. This however was not possible. Because our network policy doesn’t allow for multiple log ins. All is centrally stored on some M$ domain controllers. That’s why SVN over HTTP was used in the first place. End of story i thought.

But i couldn’t let go of this. The situation becomes less workable every day. So i decided to look around for other options. When i notice some log ins were merged to the *nix boxes. i immediately thought about using SSH in combination with SVN. Logging in on the *nix box was no problem. So let’s try that with SVN.

svn+ssh://location/of/repo

This gives an error about a mismatched handshake. And i just couldn’t get this to work until i found a post made by Martin Woodward. He explains how to setup an environment variable on my local windows machine. To the tortoise PLink executable.

Set the environment variable (by right-clicking on My Computer, Properties, Advanced, Environment Variables, New):-
Variable name: SVN_SSH
Variable value: C:\\Program Files\\TortoiseSVN\\bin\\TortoisePlink.exe

This seemed to work i thought. i was presented with a login box. But no matter how i tried i couldn’t login. It just kept giving me the login box. So i opened a SSH session to the server and tried to do the same from there.

svn list svn+ssh://location/of/repo

After hitting enter i typed my password and got the following message.

bash: line 1: svnserve: command not found
svn: Connection closed unexpectedly

Now we’re getting somewhere. So svnserve could not be found. Probably because it’s not in the $PATH variable. So let’s add it there.

vi .bashrc
PATH=$PATH:/usr/local/bin

So let’s try to get a listing of the repo.

svn list svn+ssh://location/of/repo

This gave me the following message:

svn: No repository found in ‘svn+ssh://location/of/repo’

Seems that when you use SVN over SSH you need to give the full location to the repository. Not the web server path but the complete file system path. After changing the webserver path for the file system path everything worked out. From here on adding the repository to eclipse is easy.
I always open the repository view. From there i right click > New > repository Location. The URL to add will look something like

svn+ssh://file/system/location/of/repository

So now we can enjoy the speed of the svn:// protocol. And the security of SSH. So let’s do some coding :)

When i first started out as a web developer. i didn’t really mind about structure. Almost every project had different directory structure. And namespaces i never heard off. I noticed that the longer i write code. The more I’m looking for a basic structure to start from. This can be the structure of a web application / site. but also the structure inside the SVN repository.

So today i took a bit of time to think about a good structure for our new SVN repository here at the office. After a bit of browsing the web. Looking at other proposals. And looking around our old SVN repository. I came up with a new structure.

Our SVN repository will start off with three directories.

– Development
– Design
– ??

The Development directory is what i will be talking about. I can’t talk for the designers. Because.. well i’m not one of them. Although i did a lot of design work in the past. The ?? directory is for another part of our department. So let’s see what i came up with.

Inside the Development directory we start of with two directories.

– Projects
– Library

Projects:
This directory will contain all projects. With projects i mean web applications / sites. All these projects will adhere to a basic structure. Which i will talk about a bit later. Besides the same basic structure. Every projects has the trunk, tags and branches sub directories. If a project contains “sub” projects. The trunk, tags and branches will shift one level deeper in the tree. Below is a small example.

– Project_1
—- trunk
—- tags
—- branches
– Project_2
—- www
—— trunk
—— tags
—— branches
—- intranet
—— trunk
—— tags
—— branches

Library:
The library directory will contain all shared libraries. This can be third party libraries or internally developed libraries that will be used in multiple applications. Below is a small example. I’m not sure about the trunk, tags, branches directories yet. For the internally developed libraries we will implement this structure. For the third party libraries we won’t. I think :)

– Zend
– DB
—- trunk
—- tags
—- branches

Now this will cover most of the project parts. What’s left now is defining a basic structure for the projects that will ive inside the repository and on the web servers. For a basic structure i’m inspired by the layout of an Zend Framework MVC style project. Why? i already work with this structure for a while now. outside and inside the Zend Framework projects. It separates the code from the views / templates. It makes sure the code is outside the web root. And it just looks clean.

So for our projects i came up with this structure.

The structure really consists of three parts. The first is “public”. This directory contains all data that should be globally accessible. We can think about images, stylesheetes, javascript libraries, etc.

– public
—- images
—- css
—- scripts
—- index.*

The application directory contains all the real code and views / templates. All views / templates will be stored in the views directory. All project specific code will be placed in the library directory. And if the project is MVC based. It will use the controllers directory to store the application controllers.

– application
—- controllers
—- library
—- views

The final part of the structure is a collection of directories that can but are not always used in a project.

The logs directory can be handy for logging data of the application. But can also be used to store php error logs or maybe even Apache logs. The cache directory is there to store cached files. The structure inside this directory will depend on the cache mechanism used in the project. The files directory contains files that are somehow linked to the project but have no place in any other directories. Some test scripts or tools can be placed there. And last but not least the docs directory. Which will contain all project documentation.

– logs
– cache
– files
– docs

I wanted to post this yesterday. But i just couldn’t find any time for it. Yesterday i showed the initial proposal to the rest of the team. But we didn’t decide yet. What the final structure will be.