Web Development and stuff…

I was kinda bored yesterday. And was poking around the SVN hooks scripts. When i noticed the post-commit script i was using was written in Python. Nothing wrong with that. I just wondered how hard it would be to do the same in PHP. So i came up with a little test. Which at this moment lacks error checking and just looks plain ugly. But it does the job. and i only needed a working PHP CLI executable to make it work.

First we set the email address and the location of svnlook.

#!/usr/bin/php
<?php
error_reporting(E_ALL | E_STRICT); 

$address = 'some email address';
$svnlook = '/usr/bin/svnlook';

Use the argv array to get to the commandline parameters and build up a SVN statement to get some basic info.

$repository = $_SERVER['argv'][1];
$revision = $_SERVER['argv'][2];

$cmdInfo = "${svnlook} info ${repository} -r ${revision}";

$output = array();
exec($cmdInfo, $output);

$author = $output[0];
$date = $output[1];
$comment = $output[3];

Now let’s get the changes made in this revisions.

$output = array();
$cmdChanges ="${svnlook} diff ${repository} -r ${revision}";
exec($cmdChanges, $output);

Once we have the diff contents we can loop through the lines and use some simple regex to highlight the changes. I’ve added a bit more code to add pre tags for a better visual effect.

$patch = '<pre class="codeBlock">';
$lineCount = count($output);
for ($x=0; $x<=$lineCount; $x++) {

    if ($x > 0 && strstr($output[$x], 'Modified:')) {
        $patch .= '</pre><pre class="codeBlock">';
    }

    $textColor = 'normal';
    if (preg_match('/^\+/', $output[$x])) {
        $textColor = 'positive';
    }
    else if (preg_match('/^-/', $output[$x])) {
        $textColor = 'negative';
    }

    $patch .= '<span class="' . $textColor . '">' . htmlentities($output[$x]) . "</span><br>";

    if ($x == $lineCount) {
        $patch .= "</pre>";
    }
}

Now we can build the body for the email that will be generated.

$body .= "Author : " . $author . "<br>";
$body .= "Date : " . $date . "<br>";
$body .= "Comment : " . $comment . "<br><br>";

$body .= $patch;

Add a MIME header so my mail client displays it properly. And send it of through the build in mail() function.

$headers  = 'MIME-Version: 1.0' . "\r\n";
$headers .= 'Content-type: text/html; charset=iso-8859-1' . "\r\n";

$bodyHTML = file_get_contents("/path/to/some/HTML/template");
$bodyHTML = str_replace('{EMAIL}', $body, $bodyHTML);

mail($address, 'SVN post-commit', $bodyHTML, $headers);

Yesterday my colleague was complaining he couldn’t do a normal check out of one of the projects in Subversion. At first i thought it was just something wrong with his local settings. Because i could do the check out just fine. Today when i was thinking about this problem. I quickly realized we are on two different platforms. He works on Windows while I’m on Linux.

And since i had to do some Photoshop work today. I took the chance to install Subversion on my Windows partition. When i did a checkout of the failing project. I ran into the same problems as my colleague did before. Two different errors were preventing me from making a complete checkout. The first error:

svn: In directory ‘path/to/some/images’
svn: Can’t copy ‘path/to/some/images/image.JPG.svn-base’ to ‘path/to/some/image.JPG.tmp’: No such file or directory

Is caused by multiple files with the same name. At  least to windows it looks like this. Because image.jpg and image.JPG are two different files on Linux. On windows however these two files are one and the same. This causes Subversion to choke. After manually deleting all duplicates from the repositorie it self the problem dissapeared.

The second error was something like:

svn: Working copy ‘/some/path/\ ‘ is missing or not locked

This one was actually pretty obvious. A file name with an illigal character. I wonder how it got in the repository. The folder name causeing the problems was a folder called ‘\ ‘. Which was empty. After removing this. I could do a complete checkout again.

This should be pretty damn straight forward. However i ran into some problems while setting this up. At my daily work spot we use SVN over HTTP so we can do NTLM authentication. This sucks in many ways. But most of all it’s slow. And i mean really slow. Last week i tried to retrieve the log information for a project in the repository. This took my IDE 30 minutes. At first i thought my IDE was just being lame on me. But it turns out it’s SVN over HTTP that’s the culprit.

So i started to do some testing over the standard svn:// protocol. And retrieving the logs on the same repository took about 2 seconds. Amazing. That’s enough reason to switch to the svn:// protocol. This however was not possible. Because our network policy doesn’t allow for multiple log ins. All is centrally stored on some M$ domain controllers. That’s why SVN over HTTP was used in the first place. End of story i thought.

But i couldn’t let go of this. The situation becomes less workable every day. So i decided to look around for other options. When i notice some log ins were merged to the *nix boxes. i immediately thought about using SSH in combination with SVN. Logging in on the *nix box was no problem. So let’s try that with SVN.

svn+ssh://location/of/repo

This gives an error about a mismatched handshake. And i just couldn’t get this to work until i found a post made by Martin Woodward. He explains how to setup an environment variable on my local windows machine. To the tortoise PLink executable.

Set the environment variable (by right-clicking on My Computer, Properties, Advanced, Environment Variables, New):-
Variable name: SVN_SSH
Variable value: C:\\Program Files\\TortoiseSVN\\bin\\TortoisePlink.exe

This seemed to work i thought. i was presented with a login box. But no matter how i tried i couldn’t login. It just kept giving me the login box. So i opened a SSH session to the server and tried to do the same from there.

svn list svn+ssh://location/of/repo

After hitting enter i typed my password and got the following message.

bash: line 1: svnserve: command not found
svn: Connection closed unexpectedly

Now we’re getting somewhere. So svnserve could not be found. Probably because it’s not in the $PATH variable. So let’s add it there.

vi .bashrc
PATH=$PATH:/usr/local/bin

So let’s try to get a listing of the repo.

svn list svn+ssh://location/of/repo

This gave me the following message:

svn: No repository found in ’svn+ssh://location/of/repo’

Seems that when you use SVN over SSH you need to give the full location to the repository. Not the web server path but the complete file system path. After changing the webserver path for the file system path everything worked out. From here on adding the repository to eclipse is easy.
I always open the repository view. From there i right click > New > repository Location. The URL to add will look something like

svn+ssh://file/system/location/of/repository

So now we can enjoy the speed of the svn:// protocol. And the security of SSH. So let’s do some coding

When i first started out as a web developer. i didn’t really mind about structure. Almost every project had different directory structure. And namespaces i never heard off. I noticed that the longer i write code. The more I’m looking for a basic structure to start from. This can be the structure of a web application / site. but also the structure inside the SVN repository.

So today i took a bit of time to think about a good structure for our new SVN repository here at the office. After a bit of browsing the web. Looking at other proposals. And looking around our old SVN repository. I came up with a new structure.

Our SVN repository will start off with three directories.

– Development
– Design
– ??

The Development directory is what i will be talking about. I can’t talk for the designers. Because.. well i’m not one of them. Although i did a lot of design work in the past. The ?? directory is for another part of our department. So let’s see what i came up with.

Inside the Development directory we start of with two directories.

– Projects
– Library

Projects:
This directory will contain all projects. With projects i mean web applications / sites. All these projects will adhere to a basic structure. Which i will talk about a bit later. Besides the same basic structure. Every projects has the trunk, tags and branches sub directories. If a project contains “sub” projects. The trunk, tags and branches will shift one level deeper in the tree. Below is a small example.

– Project_1
—- trunk
—- tags
—- branches
– Project_2
—- www
—— trunk
—— tags
—— branches
—- intranet
—— trunk
—— tags
—— branches

Library:
The library directory will contain all shared libraries. This can be third party libraries or internally developed libraries that will be used in multiple applications. Below is a small example. I’m not sure about the trunk, tags, branches directories yet. For the internally developed libraries we will implement this structure. For the third party libraries we won’t. I think

– Zend
– DB
—- trunk
—- tags
—- branches

Now this will cover most of the project parts. What’s left now is defining a basic structure for the projects that will ive inside the repository and on the web servers. For a basic structure i’m inspired by the layout of an Zend Framework MVC style project. Why? i already work with this structure for a while now. outside and inside the Zend Framework projects. It separates the code from the views / templates. It makes sure the code is outside the web root. And it just looks clean.

So for our projects i came up with this structure.

The structure really consists of three parts. The first is “public”. This directory contains all data that should be globally accessible. We can think about images, stylesheetes, javascript libraries, etc.

– public
—- images
—- css
—- scripts
—- index.*

The application directory contains all the real code and views / templates. All views / templates will be stored in the views directory. All project specific code will be placed in the library directory. And if the project is MVC based. It will use the controllers directory to store the application controllers.

– application
—- controllers
—- library
—- views

The final part of the structure is a collection of directories that can but are not always used in a project.

The logs directory can be handy for logging data of the application. But can also be used to store php error logs or maybe even Apache logs. The cache directory is there to store cached files. The structure inside this directory will depend on the cache mechanism used in the project. The files directory contains files that are somehow linked to the project but have no place in any other directories. Some test scripts or tools can be placed there. And last but not least the docs directory. Which will contain all project documentation.

– logs
– cache
– files
– docs

I wanted to post this yesterday. But i just couldn’t find any time for it. Yesterday i showed the initial proposal to the rest of the team. But we didn’t decide yet. What the final structure will be.