$matches = array();
preg_match("/^Get(.+)Repository$/", "GetFooBarRepository", $matches);
var_dump($matches);

array(2) {
[0] => string(19) “GetFooBarRepository”
[1] => string(6) “FooBar”
}

And there is nothing wrong with that. Except for the fact that numeric indexes are not always easy to work with. And it does not look all that clean in the code itself. SO last week my LD pointed out the fact that the likes using name based indexes for the matched patterns. And this is pretty sweet. I have seen it before but never bothered to adopt it myself. And the result looks like

$matches = array();
preg_match("/^Get(?.+)Repository$/", "GetFooBarRepository", $matches);
var_dump($matches);

array(3) {
[0] => string(19) “GetFooBarRepository”
‘repositoryName’ => string(6) “FooBar”
[1] => string(6) “FooBar”
}

And i have to say. It looks a lot cleaner. So i added this to my bag of tricks. And will be using it from now on.

$url = reset($file->getPaths());

So how can that throw an error like “Only variables should be passed by reference” you might ask? Well as it turns out to be. PHP functions don’t like arguments returned from another function. As with this case. The ->getPaths() method holds a reference to the returned array. Which it shouldn’t but that’s another thing.

So i guess from now i will doing the thing below. Or check for references beforehand!

$paths = $file->getPaths();
$url = reset($paths);

Last week a critical bug was discovered in PHP. Which affects versions 5.3.9 and 5.2.17. The bug could be exploited to run arbitrary code on a remote PHP system. So upgrade your systems. And of course Stefan Esser popped up with some wise words :)… O well i still think the guy does great work.

More info about the issue can be found on packetstorm (CVE-2012-0830)

session adoption & session fixation

The problem exists because the current session module does not validate the session id that comes in from a cookie. This means uninitialized session id’s can be passed by the client. This happens due to the fact that browsers overwrite cookie if multiple cookies are send per request.
Some people would say this is solvable by implementing session_regenerate_id(). But this is not the case.

Because session fixation can be used to take over control of web applications. Validation is required when multiple cookies are send per request. When multiple cookie are send with a request. Browsers send multiple cookies without domain / path information. This way it’s impossible to tell which cookie belongs to which domain.

So how do we fix this?

There is some userland code that does offer the ability to validate session data. But this has not been widely adopted by other developers.

Code that adds the session ID as a validation key:

session_destory();
session_regenerate_id();
$_SESSION['valid_id'] = session_id();

And the code to check if the session was properly initialized:

if ($_SESSION['valid_id'] !== session_id()) {
  die('Invalid use of session ID');
}

Thank god the internal developer know this. And are working to fix this. For the past days there has been an interesting discussion going on on the internals list. About applying a patch that will fix this. The patch will add some new php.ini features and a new method validate_id() for the session save handler. Hopefully this will be available in version 5.4.

To not break BC strict_mode will be disabled by default. But can be enabled by setting the following setting in php.ini. When enabled uninitialized session ID will be discarded.

session.use_strict_mode=0

To prevent a DoS instead of session fixation. An new feature has been added that deletes possible malicious cookies that prevent new session ID.

session.safe_session_cookie=1

You can read more about session fixation and the upcoming patch on the PHP-Wiki

Intrigued by the fact that my 32-bit Ubuntu installation did not suffer from the same issue. I started to read the PHP5 changelog for Ubuntu.

CFLAGS=”-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64″ ./configure

But this has a negative side effect. Namely PHP running 50% slower then it should be. And to my surprise Ubuntu applies the CFLAGS above when compiling PHP5 for a 32-bit platform. So that would mean PHP on Ubuntu is always running 50% slower? Well according to this is should be the case.

I downloaded php-5.3.8 from php.net and compiled two versions. One with the CFLAGS set for large files. And one normal without any changes. After that i downloaded the benchmark script from php-benchmark-script.com. And did a couple of runs on each of the two installs. The results are stunning.

PHP-5.3.8 compiled with large file support:

test_math : 4.414 sec.
test_stringmanipulation : 4.968 sec.
test_loops : 3.529 sec.
test_ifelse : 2.344 sec.
————————————–
Total time: : 15.255 sec.

PHP-5.3.8 compiled without large file support:

test_math : 2.274 sec.
test_stringmanipulation : 2.286 sec.
test_loops : 1.619 sec.
test_ifelse : 1.228 sec.
————————————–
Total time: : 7.407 sec.

That’s pretty much a 50% speed decrease. One more thing i tried is adding the ‘AC_SYS_LARGEFILE’ macro to configure.in and rebuild the configure script. But this had no effect at all.

$ cd php-5.3.8
$ vi configure.in (add AC_SYS_LARGEFILE somewhere)
$ export PHP_AUTOCONF=/usr/share/autoconf2.59
$ ./buildconf –force

I then tried two things. First i build without the CFLAGS. But this didn’t seem to do much. Then i used the same configure script with the CFLAGS for large file support. But there was no speed increase measurable.

So why would Ubuntu have made the choice to compile PHP with the large file support on a 32-bit platform? Are there really that much developers that work on large files in web / cli applications written in PHP? Enough to sacrifice a 50% speed decrease?

I don’t really know the answer to that. But i will do my large file processing on 64-bit machines. And will compile PHP from scratch from now on. Until i have upgraded my aging hardware.

Importer found (0) files to import!

Hmmm. That’s not right. So i had a look at the code behind the importer. Which basically is a loop using a DirectoryIterator object. And some var_dump calls revealed the issue. For some reason ->isFile() was returning (false) for regular files. WTF! Let’s test that on the command line.

$ php -r “var_dump(is_file(‘/some/file.ext’));”;
bool(false)

Ok so we have an issue here. How big are these files really. A inspection revealed they are all over 2GB. Maybe some 32 bit issue? As the platform the code is running on is a 32 bit server. So i asked my colleagues, Googled a bit and read through php.net. To find out that there is an issue with PHP and files larger then 2GB.

https://bugs.php.net/bug.php?id=27792
https://bugs.php.net/bug.php?id=48886
http://nl.php.net/manual/en/function.filesize.php

Those however all seem related to filesize. The filesize function manual page even has a note about it. Maybe it’s related?

Note: Because PHP’s integer type is signed and many platforms use 32bit integers, filesize() may return unexpected results for files which are larger than 2GB. For files between 2GB and 4GB in size this can usually be overcome by using sprintf(“%u”, filesize($file)).

But i can’t apply that patch on a production server. So i came up with a simple solution for now. I extended the DirectoryIterator class and have overwritten the isFile method. Which works for now (don’t think this will work on windows).

Class MyDirectoryIterator extends DirectoryIterator {
	public function isFile() {
		return (integer) exec("[ -f {$this->getPathname()} ] && echo 1 || echo 0");
	}
}

Convinced it was a 32 bit issue. I came home later that day. And wanted to try it out on my own desktop. That is a 32 bit system and runs Ubuntu 11.04. To my surprise the result was different then i expected.

$ php -r “var_dump(is_file(‘/some/file.ext’));”;
bool(true)

I used the same files as before. And tested some more big files. But the result was the same. Weird. Let’s try some other 32 bit machines.

Ubuntu 11.04: bool(true)

CentOS release 5.6 (Final): bool(false)
Debian 6.0.2 (squeeze): bool(false)

Only my desktop at home seems to have a good result. Ubuntu must have some patch somewhere to fix this issue? To confirm i compiled PHP 5.3.8 from source. And did the same test again on Ubuntu 11.04. And this time it was (false).

$ php -r “var_dump(is_file(‘/some/file.ext’));”;
bool(false)

I am not really in the mood to search the Ubuntu changelog. And for now the work around will do. But i really would like to know what patch is applied to resolve the issue.

[ update ]

While applying the patch for the is_file issue. I was confronted with the fact that way more function calls cause issues. So while waiting for PHP to get patched i had to create some workarounds for the time being.

Getting the filesize:

(integer) exec("stat -c%s {$file->getFilename()}");

Calculate a MD5 checksum:

$md5 = exec("md5sum {$file->getFilename()}");
$expl = explode('\t', $md5);
return (string) $expl[0];

Calculate the CRC32 checksum:

$hash = exec("cksum {$this->path}");
$expl = explode(' ', $hash);
return $expl[0];

Get the modified time:

$stat = explode('.', exec("stat -c%y {$this->path}"));
$timestamp = strtotime($stat[0]);
return $timestamp;

Hopefully that will do for now. On a side note the issue is solvable by setting certain CFLAGS when compiling PHP. I have no idea what the impact of that will be on the PHP binary. But it does seem to solve the issue. Not sure how one would apply that when PHP is installed from the distro’s repository though.

CFLAGS=”-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64″ ./configure

So the code i was working on. Did something like this.

$date = date('D M d H:i:s Y', strtotime($someVar));

My thought was that by setting the correct locale the dates would appear in the correct language. Wrong!

date_default_timezone_set('Europe/Amsterdam');
setlocale(LC_ALL, 'nl_NL.utf8');

After a reload i was greeted by the same dates as before. In plain English. Oke no worries. Let’s see what setlocale returns.

var_dump(setlocale(LC_ALL, 'nl_NL.utf8'));

bool(false)

That’s not good. Seems like we are missing some locales on the server. Let’s check.

locale -a

en_AG
en_AG.utf8
en_AU.utf8
en_BW.utf8
en_CA.utf8
en_DK.utf8
en_GB.utf8
en_HK.utf8
en_IE.utf8
en_IN
en_IN.utf8
en_NG
en_NG.utf8
en_NZ.utf8
en_PH.utf8
en_SG.utf8
en_US.utf8
en_ZA.utf8
en_ZW.utf8
…

And some more output after that. But not the one i am looking for. But thankfully aptitude was kind enough to provide the missing language packages.

nl_NL
nl_NL@euro
nl_NL.iso88591
nl_NL.iso885915@euro
nl_NL.utf8

So let’s set the correct locale for this script.

setlocale(LC_ALL, 'nl_NL.utf8');

But still no changes. I must be missing something….. Let’s consult the manual. The last line in the examples section is what i was looking for

To format dates in other languages, you should use the setlocale() and strftime() functions instead of date().

Duuh! Completely forgot about strftime. Let’s change the code.

strftime('%a %b %d %H:%M:%S %Y', strtotime($somevar));

ma aug 15 14:55:06 2011

Perfect. That did it.

Reserved keyword

So i have been banging my head over this issue for way to long today. And had to find out the hard way it was a simple little thing as usual. But Doctrine (version 1 if anybody asks) didn’t offer me any good info with the error message it was displaying. Nor did PHP or MySQL (should be MariaDB if you ask me).

While working on a very heavy Doctrine based project. I ran into a at first glance weird error. Yesterday i created a class structure. All good and well. The class i was working on was called Shows. But it represented a single Show entity and therefore should have been called Show. So i decided to do a rename action. Pretty complex because of different dependencies. But after a few tries i got it right. Except Doctrine was failing with a syntax error. After a code inspection i still couldn’t spot the issue. The only thing i didn’t do yet was output the query and run it from a shell. And this failed with the same syntax error message. Damn!

INNER JOIN show s ON …

Checked the whole query. Still couldn’t spot the issue. Then for some weird reason i decided to escape the table name with backticks `Show`. Everything worked again. Wow! So Doctrine doesn’t escape table names. But why did it fail in the first place? By now i had the feeling i was using a reserved keyword. And a check confirmed this. An error would have been helpful here. I had a bit of luck though. As it turned out the table should have been named different anyway. I made a small workaround for it in my code. But decided that was not the way to go So the lesson of the day for me was don’t use reserved words for class / table names. A good thread on the issue can be found here.

APC object corruption

The second issue to bite my ass was a nice one as well. And this time no indication of an error at all. While working on the same classes discusses shortly above. I renamed the class names and the file names on disk. And after a page reload in the browser. The framework was complaining it couldn’t load the changed class. I checked the class and file names and all were correct. After toying with it for 20 minutes i decided to ask my colleague. And he suggested to restart Apache. for the second time today i was stumped. Everything worked again. Turns out that APC couldn’t find the already in memory loaded class. After renaming the corresponding filename (seems logical now that i think of it). A restart of Apache flushed the APC cache and therefore the issue. This sounds like a bug though. And requires some further inspection or a bug report. We will see.

PHP Fatal error

Last week while setting up PHPUnit i ran into a weird issue while testing PHPUnit from a shell on my local desktop. The whole test suite would run from start to end. But at the end would display the following not so descriptive error message.

PHP Fatal error: Exception thrown without a stack frame in Unknown on line 0

This kept me busy for some hours. This happened somewhere in a large framework. And there was no clear sign of what was causing the issue. Some googling did reveal some cases in which this error may occur.

There was an exception while handling an exception
There was an exception while running a destructor
There was an exception while closing the session
There was an exception while running a shutdown function
There was an exception while running a autoload function

The issue in my case was the fact that the configuration file wasn’t loaded yet. And therefore the log file location was not set correctly. Logging in this framework is done in a plugin system which is pretty cool. We can basically assign plugins for certain log levels (and more). This was the FileLogger. This all still does not explain why it failed with this particular error message. But a bit of digging in the Log/Plugin system revealed the issue.

register_shutdown_function(array(__CLASS__, 'Shutdown'));

Message are stored in a queue and written to file in a register shutdown function. Which was failing because the logfile location was not set correctly. This was causing an exception to be thrown inside the registered shutdown function. And therefore triggering this horrible error message. Be careful with register_shutdown functions.

The kick off for the DPC11 were the tutorial days. And they had some pretty cool tutorial lined up for this year. But as always you have to choose. So me and Geoff chose to visit two of the tutorials we would benefit from the most. Since we are in pretty heavy database driven environment. We chose:

Optimizing MySQL Essentials

This session was presented by Ligaya Turmelle & Raymond DeRoo, And these two have a great chemistry between them. All the ingredients for a good sessions. And a good session it was. Even though i missed the larges part of the first 1.5 hours because i got stuck in traffic.

The second part of the tutorial was aimed at profiling and optimizing slow queries. It was Ligaya time to talk and Raymond took his stand on the soapbox a few times to makes things more clear. Or to interact with the audience in his special way. The session mainly describe how to enable and configure the MySQL slow query log. And what to look for once that is up and running. Ligaya continued with a real world example from joined.in. And show how the medium sized queries could be optimized quite a bit. Although not completely finished. The result was quite nice.

After talking about the slow query log we went into a bit more detail as to how to do query optimization. And what tools to use for the job. MySQl comes with some nice features that can be used in this process. The basics to get more information about your queries

* show create
* table status
* show indexes
* explain [ extended ]

I made quite a few notes during this sessions. And learned some cool new things. Some of the main items for me were

If possible move where clauses inside the join statements
Don’t use subselects. But if you really have to use them in the `AS` form

SELECT
t.id,
(SELECT t2foo FROM table t2 WHERE t2foo = t.bar) AS foo
FROM table t

Indexing key order is very important
Use a sequence table to generate date based reports

Doctrine 2

I was really looking forward to the Doctrine 2 session with Juozas Kaziukėnas. And it was a good session as well. Although a bit fast. And i had some issues getting Doctrine to work on my laptop. But because we make very heavy use of Doctrine 1. I was very interested in what version two has to offer over version 1. And i think Juozas did a great job in explaining the structure of Doctrine 2 and the most basic inner workings.

I was very happy to see the new Doctrine 2 models are completely decoupled from the Doctrine base classes. This is awesome. And makes for a way more portable application then with the Doctrine 1 base. Which basically ties you into using Doctrine forever.

That was tutorial day
That was day one. I had fun although quit painful with my current injury. I really enjoyed the sessions. And on our way out we were presented with a nice surprise. A copy of “Real-World Solutions for Developing High-Quality PHP Frameworks and Applications” by Sebastian Bergmann and Stefan Priebsch. A great addition to my private library. And to top things of we all got a 20% discount on the 5.3 ZCE exam.

Next up, Day 2

I missed the opening talk by Harry Verveer which is a shame. Geoff told me it was a very good inspiring and energetic talk. It got people enthusiastic. That’s what we need. At arrival it was visible that there were quite some developers here. It’s always a big difference with the tutorial day. Like the first day i missed a big part of the morning due.. to well you know. But i attended 4 cool sessions non the less.

Cooking up your Development Environment by Alistair Stead

This session was a bit different then what i envisioned. But it turned out to be a gem. Most of the talk was aimed at explaining how to setup and work with the Chef application. Before going into explaining chef. Alistair mentioned some other solutions like Capistrano, Puppet or a mix of Bash scripts.

And then he continued about explaining his day to day work day. And how he would setup his virtual work environment each morning by running a simple command that generates a fresh virtual machine to work with. Then he went into detail how Chef Solo (Distributed infrastructure management) and Chef Server (Centralized infrastructure management) work together. And how you can setup and deploy packages on the virtual machines and basically how to manage your instances from a cool and simple web GUI.

He went on explaining this hooks into Amazone’s AWS and Rackspace cloud services as well. So skies the limit. An awesome tool.

Simplify your External Dependency Management by Stephan Hochdörfer

This session was aimed at working better with project dependencies. I was quite interested in this talk. Since we work with svn:externals. Which at times can be quite a pain. Specially when the amount of applications build on a single framework tend to grow. And BC breakage is a sin.

I was a bit sad to see that this session didn’t really target managing the local dependencies. But was more aimed at handling the dependencies when deploying packages on servers. Until now this has never been an issues. Even though i was a bit disappointed. The sessions did have some valuable information.

And went into some detail how to setup an own PEAR channel to deploy packages but also touched the Java system Maven. Which i did work whit a few years back.

Clean PHP by Sebastian Bergmann

I expected more from this sessions. And i had the feeling Sebastian was blasting through the talk as if he was out of time. It went very fast. And basically was run down of all the developer principles, acronyms, etc. A good talk. But i expected more from the father of PHPUnit.

Advanced OO patterns by Tobias Schlitt

The best session of the day. Tobias is a real speaker. One that say the right things. And makes the audience laugh. I really enjoyed listening to to him talk. But a subject like Advanced OO patterns can only be cool. So i guess i he chose the right talk to give. Like most talkers he gave some props to Martin Fowler and the Gang of Four. For writing on patterns and refactoring. And basically it’s a refreshment of studying design patterns. Which i do an a daily basis.

There was one real gem however in this session that i hadn’t come across yet. Which basically is a way to do lazy loading with objects. This is nothing new. But the framework he used for the talk. Used the new 5.3 closures for this. And i really liked that. So besides the fact that i learned a cool pattern. I also learned a new way of using closures in PHP 5.3.

A great conference

That basically was the conference for me. I really was looking forward to listing in on Derick’s talk about profiling PHP applications. But due to my condition i had to stay home the last day and miss this talk. A bit sad. Thankfully he has pretty detailed PDF up on his site about the talk.

Once again IBuildings did a great job in organizing this event. Thanks guys. You once again did an awesome job. I do have some small notes however. During the sessions i heard a lot of speakers say. This is a sessions for 60 minutes. And i only have 45 minutes. So i have to skip some stuff. I mean come on how long was it known that this event was going to happen. Either give the speakers more time. Or speakers adjust your talks.

Besides the fact that i sometimes had the feeling i was visiting the IBuildings headquarters. It was a great event. Hope to see you there next year.

We need to have vim installed of course.

$ sudo aptitude install vim

Vim doesn’t seem to create the user folder automatically. So we need to create these our self. We’ll start off with a folder for plugins, color schemes and syntax settings.

$ mkdir ~/.vim
$ mkdir ~/.vim/{syntax,colors,plugin}

Most of the time i will be editing .php files. So i want some good syntax support for that. Vim has a nice plugin system. And there does seem to be a syntax file for PHP available. It’s a bit outdated. But will do for now. The only thing to do to enable it is copy it to the plugin directory.

$ mv php.vim ~/.vim/syntax

Now we have a very powerful but basic PHP editor. That’s nice and all. I would like to at least have some sort of a project manager. And there is a plugin for that as well. Download & copy!

$ mv project.vim ~/.vim/plugin

Ok that’s pretty cool. But it still looks like crap. Time for some theme sugar. It actually took me quite some time to find a theme i like. Guess i am picky. The theme / colour scheme i settled for is called wombat. The original file didn’t do much for me. But i found a good 256 colour version.

$ mv wombat256mod.vim ~/.vim/colors

That’s starting to look like a pretty nice setup. Time to make some settings permanent and to enable some other features.

$ vim ~/.vimrc

And add some settings

set rule
set wrapscan
set number
set backspace=start,indent,eol

set t_Co=256
colorscheme wombat256mod

autocmd FileType php set omnifunc=phpcomplete#CompletePHP

In the meantime i found this interesting post on Matthew Weier O’Phinney’s blog about using ctags , VIM and PHP. That looks like interesting stuff. So let’s add it. With ctags it’s possible to scan a source tree. And compile a sort of index file that vim can use for lookups.

$ sudo aptitude install exuberant-ctags
$ mkdir ~/.vim/tags

Creating an index file is a piece of cake. Move to a source directory and issue the following command.

$ cd /some/path
$ ctags-exuberant -R -f ~/.vim/tags/filename -h “.php”
–exclude=”.svn” –totals=yes
–tag-relative=yes –PHP-kinds=+cf
–regex-PHP=’/abstract class ([^ ]*)/\1/c/’
–regex-PHP=’/interface ([^ ]*)/\1/c/’
–regex-PHP=’/(public |static |abstract |protected |private )+function ([^ (]*)/\2/f/’

This will scan the whole directory structure recursively while using the regular expressions to extract useful information from the source files it finds. When done and when the parameter –totals=yes is used the following output is displayed. But probably with different numbers.

2375 files, 466018 lines (14628 kB) scanned in 11.7 seconds (1246 kB/s)
199296 tags added to tag file
199296 tags sorted in 0.00 seconds

Time to enable tags. I want to use multiple files. Some standard files for common frameworks / libraries. And a tag file per project.

$ vim ~/.vimrc
set tags=tags;~/.vim/tags/filename

So let’s start vim and have a look at what we have so far. The first thing we do when vim has started is launch the :Project plugin.

$ vim
:Project

The project plugin took some time to get used to. But it’s a very convenient plugin. And easy to get the hang of. It’s possible to create the project settings by hand. But the plugin comes with a build in tool for this. While in :Project mode and hitting \C will make it easy to create new projects.

$ vim
:Project
\C

This will guide you through the process with a few short and simple questions.

Enter the Name of the Entry: [ project name ]
Enter the Absolute Directory to Load: [ absolute path ]
Enter the CD parameter: [ path to move into ]
Enter the File Filter: [ file filter : *.php ]

When that’s done the screen will be in edit mode inside the .vimproject file. Displaying something like this.

Project name=/path/to/project CD=/path/to/project filter=”*.php” {
filename.ext
folder———————————————————-
}

It’s possible to alter the project setup here. Like adding more files. Or even adding files from other locations outside of the project.

Project name=/path/to/project CD=/path/to/project filter=”*.php” {
filename.ext
folder———————————————————-

External=/some/other/path CD=/some/other/path filter=”*.php” {
filename.ext
}
}

When done editing. Just do like normally in vim.

:w

Pretty cool we have a project browser on the left. That is easy to navigate with the arrow keys. And a work canvas on the right.

Time to test the ctags created index files. Place the cursor at the beginning of a string that you want to lookup. And hit CTRL + ] and CTRL + T to get back. Another cool way to open the source file is by using horizontal split windows by using CTRL + W ].

Testing auto-complete is just as easy. Type some php core function name partly str_ and hit CTRL + x and CTRL + o. This will display the following scrollable drop down..

That’s about it. I had good fun exploring all the features and possibilities. And have only scratched the surface. One more thing i found on Matthew’s blog was a way to test and run PHP script from vim. Awesome stuff. You can read about it here. Although i like vim. I don’t think i will be leaving ZS any time soon.