Last week i had a blast while listening to Moxie Marlinspike’s Blackhat talk the past and future of SSL. The video is an absolute must see if you have any concern about the current state of SSL or the whole web for that matter.

As many know. There has been quite the turmoil in the SSL world lately with some big CA’s (Comodo, StartCom, Diginotar, GlobalSign) getting hacked. And rogue certificates being generated for major domains like microsoft.com, google.com, etc for who knows what type of malicious purpose. The real problem here are the centralized CA’s. For SSL to work you need to explicitly trust the major CA’s. And if one gets hacked. Well.. we know the deal by now. And the funny thing is that Moxie mentions in his talk. The whole CA wasn’t even part of the original SSL protocol. As the creator of SSL said “It’s something we through in at the end”.

But instead of only bashing the CA’s. Moxie comes with a solution to the problem. In the form of a new protocol Convergence, Which exists of a client and a server package. The client right now is a simple Firefox extension. And when installed it disables the current CA system in the browser. And will use one or more of the selected notaries instead. It even works with self signed certificates. And the back-end is a modular one. And the standard CA verification can be swapped with for instance DNSSEC based verification.

There is still a level of trust involved. But you won’t lay all your trust in one specific CA. Convergence uses notaries. Notaries are anonymous nodes that can be picked at will and can be used to verify the requested certificate. And like i said the notaries are anonymous. No more browser leakage at this point. One of th notaries will act as a bounce node to which a SSL connection is made. All other notaries are contacted through this secure connection.

I am probably not the best person to explain this all. So go ahead and listen/watch Moxie’s talk and form your own opinion. But i think everybody should install this Firefox plugin. And forget about the whole CA system. I went ahead and installed a notary node myself. Which can be found here. More information about setting up a notary node yourself can be found here

My last post was about the 32-bit integer issue i was having with PHP. And besides setting up a 64-bit server one of the solutions is to compile PHP with the CFLAGS mentioned in my previous post.

Intrigued by the fact that my 32-bit Ubuntu installation did not suffer from the same issue. I started to read the PHP5 changelog for Ubuntu.

CFLAGS=”-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64″ ./configure

But this has a negative side effect. Namely PHP running 50% slower then it should be. And to my surprise Ubuntu applies the CFLAGS above when compiling PHP5 for a 32-bit platform. So that would mean PHP on Ubuntu is always running 50% slower? Well according to this is should be the case.

I downloaded php-5.3.8 from php.net and compiled two versions. One with the CFLAGS set for large files. And one normal without any changes. After that i downloaded the benchmark script from php-benchmark-script.com. And did a couple of runs on each of the two installs. The results are stunning.

PHP-5.3.8 compiled with large file support:

test_math : 4.414 sec.
test_stringmanipulation : 4.968 sec.
test_loops : 3.529 sec.
test_ifelse : 2.344 sec.
————————————–
Total time: : 15.255 sec.

PHP-5.3.8 compiled without large file support:

test_math : 2.274 sec.
test_stringmanipulation : 2.286 sec.
test_loops : 1.619 sec.
test_ifelse : 1.228 sec.
————————————–
Total time: : 7.407 sec.

That’s pretty much a 50% speed decrease. One more thing i tried is adding the ‘AC_SYS_LARGEFILE’ macro to configure.in and rebuild the configure script. But this had no effect at all.

$ cd php-5.3.8
$ vi configure.in (add AC_SYS_LARGEFILE somewhere)
$ export PHP_AUTOCONF=/usr/share/autoconf2.59
$ ./buildconf –force

I then tried two things. First i build without the CFLAGS. But this didn’t seem to do much. Then i used the same configure script with the CFLAGS for large file support. But there was no speed increase measurable.

So why would Ubuntu have made the choice to compile PHP with the large file support on a 32-bit platform? Are there really that much developers that work on large files in web / cli applications written in PHP? Enough to sacrifice a 50% speed decrease?

I don’t really know the answer to that. But i will do my large file processing on 64-bit machines. And will compile PHP from scratch from now on. Until i have upgraded my aging hardware.

So we have a PHP based importer script that does some heavy duty media processing at the office. And i had to import some new media today. But for some reason a couple of files weren’t picked up without a message. So i cleaned up the upload folder. The only files left were the files not being processed. And when i started the importer. The result was.

Importer found (0) files to import!

Hmmm. That’s not right. So i had a look at the code behind the importer. Which basically is a loop using a DirectoryIterator object. And some var_dump calls revealed the issue. For some reason ->isFile() was returning (false) for regular files. WTF! Let’s test that on the command line.

$ php -r “var_dump(is_file(‘/some/file.ext’));”;
bool(false)

Ok so we have an issue here. How big are these files really. A inspection revealed they are all over 2GB. Maybe some 32 bit issue? As the platform the code is running on is a 32 bit server. So i asked my colleagues, Googled a bit and read through php.net. To find out that there is an issue with PHP and files larger then 2GB.

https://bugs.php.net/bug.php?id=27792
https://bugs.php.net/bug.php?id=48886
http://nl.php.net/manual/en/function.filesize.php

Those however all seem related to filesize. The filesize function manual page even has a note about it. Maybe it’s related?

Note: Because PHP’s integer type is signed and many platforms use 32bit integers, filesize() may return unexpected results for files which are larger than 2GB. For files between 2GB and 4GB in size this can usually be overcome by using sprintf(“%u”, filesize($file)).

But i can’t apply that patch on a production server. So i came up with a simple solution for now. I extended the DirectoryIterator class and have overwritten the isFile method. Which works for now (don’t think this will work on windows).

Class MyDirectoryIterator extends DirectoryIterator {
	public function isFile() {
		return (integer) exec("[ -f {$this->getPathname()} ] && echo 1 || echo 0");
	}
}

Convinced it was a 32 bit issue. I came home later that day. And wanted to try it out on my own desktop. That is a 32 bit system and runs Ubuntu 11.04. To my surprise the result was different then i expected.

$ php -r “var_dump(is_file(‘/some/file.ext’));”;
bool(true)

I used the same files as before. And tested some more big files. But the result was the same. Weird. Let’s try some other 32 bit machines.

Ubuntu 11.04: bool(true)

CentOS release 5.6 (Final): bool(false)
Debian 6.0.2 (squeeze): bool(false)

Only my desktop at home seems to have a good result. Ubuntu must have some patch somewhere to fix this issue? To confirm i compiled PHP 5.3.8 from source. And did the same test again on Ubuntu 11.04. And this time it was (false).

$ php -r “var_dump(is_file(‘/some/file.ext’));”;
bool(false)

I am not really in the mood to search the Ubuntu changelog. And for now the work around will do. But i really would like to know what patch is applied to resolve the issue.

[ update ]

While applying the patch for the is_file issue. I was confronted with the fact that way more function calls cause issues. So while waiting for PHP to get patched i had to create some workarounds for the time being.

Getting the filesize:

(integer) exec("stat -c%s {$file->getFilename()}");

Calculate a MD5 checksum:

$md5 = exec("md5sum {$file->getFilename()}");
$expl = explode('\t', $md5);
return (string) $expl[0];

Calculate the CRC32 checksum:

$hash = exec("cksum {$this->path}");
$expl = explode(' ', $hash);
return $expl[0];

Get the modified time:

$stat = explode('.', exec("stat -c%y {$this->path}"));
$timestamp = strtotime($stat[0]);
return $timestamp;

Hopefully that will do for now. On a side note the issue is solvable by setting certain CFLAGS when compiling PHP. I have no idea what the impact of that will be on the PHP binary. But it does seem to solve the issue. Not sure how one would apply that when PHP is installed from the distro’s repository though.

CFLAGS=”-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64″ ./configure

While doing some front end work yesterday. I got trapped by a jQuery issue. Well not JQuery specific. The issue was actually triggered by some other hand crafted code. Every time i would click a link inside my grid view firebug would throw an error.

uncaught exception: Syntax error, unrecognized expression: .

And the markup that triggered the error was

Nothing wrong there right? And it actually took my quite some time to figure this one out. It would be nice to have a tool that can tell you there are multiple click events assigned to a element? But for now it was just some manual searching and testing.

The issue was caused by an other snippet of Javascript code inside another .js file. This piece of code attached a click event to every div inside a grid td. Which may be a bit to greedy.

$('.admin .gridbg tr td span').click(function() {

And my link was in a nested td inside the grid. And also contained a span tag. So it was actually firing off two click events. From which one failed. Fixing it after that was easy. Either make the first click binding less greedy. Or change the markup of my second grid. I choose the last one.

About a week ago i was working on a twitter widget for a website. This required some dates to be displayed in Dutch. And i found out the hard way my knowledge on this has faded away over time.

So the code i was working on. Did something like this.

$date = date('D M d H:i:s Y', strtotime($someVar));

My thought was that by setting the correct locale the dates would appear in the correct language. Wrong!

date_default_timezone_set('Europe/Amsterdam');
setlocale(LC_ALL, 'nl_NL.utf8');

After a reload i was greeted by the same dates as before. In plain English. Oke no worries. Let’s see what setlocale returns.

var_dump(setlocale(LC_ALL, 'nl_NL.utf8'));

bool(false)

That’s not good. Seems like we are missing some locales on the server. Let’s check.

locale -a

en_AG
en_AG.utf8
en_AU.utf8
en_BW.utf8
en_CA.utf8
en_DK.utf8
en_GB.utf8
en_HK.utf8
en_IE.utf8
en_IN
en_IN.utf8
en_NG
en_NG.utf8
en_NZ.utf8
en_PH.utf8
en_SG.utf8
en_US.utf8
en_ZA.utf8
en_ZW.utf8
…

And some more output after that. But not the one i am looking for. But thankfully aptitude was kind enough to provide the missing language packages.

nl_NL
nl_NL@euro
nl_NL.iso88591
nl_NL.iso885915@euro
nl_NL.utf8

So let’s set the correct locale for this script.

setlocale(LC_ALL, 'nl_NL.utf8');

But still no changes. I must be missing something….. Let’s consult the manual. The last line in the examples section is what i was looking for

To format dates in other languages, you should use the setlocale() and strftime() functions instead of date().

Duuh! Completely forgot about strftime. Let’s change the code.

strftime('%a %b %d %H:%M:%S %Y', strtotime($somevar));

ma aug 15 14:55:06 2011

Perfect. That did it.

I was looking for the list of Memcached telnet commands. And couldn’t find much in the docs. So after some Google searching i finally found the list. And will keep it here for future reference.

So i did a bit of reading while working on a memcache tool in PHP. I found some good documentation on the memcached.org website.

What a day. Every now and then there are such days where you wished you stayed in bed. Today was one of them. And i was confronted by two weird errors with very non descriptive error message or no error message at all. In both cases the issue was solved without a real solution. This post is just to vent my frustration of the day. And now that i take the time to write this down i might as well add the issue i ran into last week. Will save that one for last.

Reserved keyword

So i have been banging my head over this issue for way to long today. And had to find out the hard way it was a simple little thing as usual. But Doctrine (version 1 if anybody asks) didn’t offer me any good info with the error message it was displaying. Nor did PHP or MySQL (should be MariaDB if you ask me).

While working on a very heavy Doctrine based project. I ran into a at first glance weird error. Yesterday i created a class structure. All good and well. The class i was working on was called Shows. But it represented a single Show entity and therefore should have been called Show. So i decided to do a rename action. Pretty complex because of different dependencies. But after a few tries i got it right. Except Doctrine was failing with a syntax error. After a code inspection i still couldn’t spot the issue. The only thing i didn’t do yet was output the query and run it from a shell. And this failed with the same syntax error message. Damn!

INNER JOIN show s ON …

Checked the whole query. Still couldn’t spot the issue. Then for some weird reason i decided to escape the table name with backticks `Show`. Everything worked again. Wow! So Doctrine doesn’t escape table names. But why did it fail in the first place? By now i had the feeling i was using a reserved keyword. And a check confirmed this. An error would have been helpful here. I had a bit of luck though. As it turned out the table should have been named different anyway. I made a small workaround for it in my code. But decided that was not the way to go So the lesson of the day for me was don’t use reserved words for class / table names. A good thread on the issue can be found here.

APC object corruption

The second issue to bite my ass was a nice one as well. And this time no indication of an error at all. While working on the same classes discusses shortly above. I renamed the class names and the file names on disk. And after a page reload in the browser. The framework was complaining it couldn’t load the changed class. I checked the class and file names and all were correct. After toying with it for 20 minutes i decided to ask my colleague. And he suggested to restart Apache. for the second time today i was stumped. Everything worked again. Turns out that APC couldn’t find the already in memory loaded class. After renaming the corresponding filename (seems logical now that i think of it). A restart of Apache flushed the APC cache and therefore the issue. This sounds like a bug though. And requires some further inspection or a bug report. We will see.

PHP Fatal error

Last week while setting up PHPUnit i ran into a weird issue while testing PHPUnit from a shell on my local desktop. The whole test suite would run from start to end. But at the end would display the following not so descriptive error message.

PHP Fatal error: Exception thrown without a stack frame in Unknown on line 0

This kept me busy for some hours. This happened somewhere in a large framework. And there was no clear sign of what was causing the issue. Some googling did reveal some cases in which this error may occur.

There was an exception while handling an exception
There was an exception while running a destructor
There was an exception while closing the session
There was an exception while running a shutdown function
There was an exception while running a autoload function

The issue in my case was the fact that the configuration file wasn’t loaded yet. And therefore the log file location was not set correctly. Logging in this framework is done in a plugin system which is pretty cool. We can basically assign plugins for certain log levels (and more). This was the FileLogger. This all still does not explain why it failed with this particular error message. But a bit of digging in the Log/Plugin system revealed the issue.

register_shutdown_function(array(__CLASS__, 'Shutdown'));

Message are stored in a queue and written to file in a register shutdown function. Which was failing because the logfile location was not set correctly. This was causing an exception to be thrown inside the registered shutdown function. And therefore triggering this horrible error message. Be careful with register_shutdown functions.

Last week was the 5th DPC (Dutch PHP Conference) in the Amsterdam Rai. And thanks to my awesome employer i once again had the chance to be there. This year with a full conference ticket (including the tutorial days). And i had a blast. I wanted to do this post earlier this weekend but due to some current injury i wasn’t able to. So here goes.

The kick off for the DPC11 were the tutorial days. And they had some pretty cool tutorial lined up for this year. But as always you have to choose. So me and Geoff chose to visit two of the tutorials we would benefit from the most. Since we are in pretty heavy database driven environment. We chose:

Optimizing MySQL Essentials

This session was presented by Ligaya Turmelle & Raymond DeRoo, And these two have a great chemistry between them. All the ingredients for a good sessions. And a good session it was. Even though i missed the larges part of the first 1.5 hours because i got stuck in traffic.

The second part of the tutorial was aimed at profiling and optimizing slow queries. It was Ligaya time to talk and Raymond took his stand on the soapbox a few times to makes things more clear. Or to interact with the audience in his special way. The session mainly describe how to enable and configure the MySQL slow query log. And what to look for once that is up and running. Ligaya continued with a real world example from joined.in. And show how the medium sized queries could be optimized quite a bit. Although not completely finished. The result was quite nice.

After talking about the slow query log we went into a bit more detail as to how to do query optimization. And what tools to use for the job. MySQl comes with some nice features that can be used in this process. The basics to get more information about your queries

* show create
* table status
* show indexes
* explain [ extended ]

I made quite a few notes during this sessions. And learned some cool new things. Some of the main items for me were

If possible move where clauses inside the join statements
Don’t use subselects. But if you really have to use them in the `AS` form

SELECT
t.id,
(SELECT t2foo FROM table t2 WHERE t2foo = t.bar) AS foo
FROM table t

Indexing key order is very important
Use a sequence table to generate date based reports

Doctrine 2

I was really looking forward to the Doctrine 2 session with Juozas Kaziukėnas. And it was a good session as well. Although a bit fast. And i had some issues getting Doctrine to work on my laptop. But because we make very heavy use of Doctrine 1. I was very interested in what version two has to offer over version 1. And i think Juozas did a great job in explaining the structure of Doctrine 2 and the most basic inner workings.

I was very happy to see the new Doctrine 2 models are completely decoupled from the Doctrine base classes. This is awesome. And makes for a way more portable application then with the Doctrine 1 base. Which basically ties you into using Doctrine forever.

That was tutorial day
That was day one. I had fun although quit painful with my current injury. I really enjoyed the sessions. And on our way out we were presented with a nice surprise. A copy of “Real-World Solutions for Developing High-Quality PHP Frameworks and Applications” by Sebastian Bergmann and Stefan Priebsch. A great addition to my private library. And to top things of we all got a 20% discount on the 5.3 ZCE exam.

Next up, Day 2

I missed the opening talk by Harry Verveer which is a shame. Geoff told me it was a very good inspiring and energetic talk. It got people enthusiastic. That’s what we need. At arrival it was visible that there were quite some developers here. It’s always a big difference with the tutorial day. Like the first day i missed a big part of the morning due.. to well you know. But i attended 4 cool sessions non the less.

Cooking up your Development Environment by Alistair Stead

This session was a bit different then what i envisioned. But it turned out to be a gem. Most of the talk was aimed at explaining how to setup and work with the Chef application. Before going into explaining chef. Alistair mentioned some other solutions like Capistrano, Puppet or a mix of Bash scripts.

And then he continued about explaining his day to day work day. And how he would setup his virtual work environment each morning by running a simple command that generates a fresh virtual machine to work with. Then he went into detail how Chef Solo (Distributed infrastructure management) and Chef Server (Centralized infrastructure management) work together. And how you can setup and deploy packages on the virtual machines and basically how to manage your instances from a cool and simple web GUI.

He went on explaining this hooks into Amazone’s AWS and Rackspace cloud services as well. So skies the limit. An awesome tool.

Simplify your External Dependency Management by Stephan Hochdörfer

This session was aimed at working better with project dependencies. I was quite interested in this talk. Since we work with svn:externals. Which at times can be quite a pain. Specially when the amount of applications build on a single framework tend to grow. And BC breakage is a sin.

I was a bit sad to see that this session didn’t really target managing the local dependencies. But was more aimed at handling the dependencies when deploying packages on servers. Until now this has never been an issues. Even though i was a bit disappointed. The sessions did have some valuable information.

And went into some detail how to setup an own PEAR channel to deploy packages but also touched the Java system Maven. Which i did work whit a few years back.

Clean PHP by Sebastian Bergmann

I expected more from this sessions. And i had the feeling Sebastian was blasting through the talk as if he was out of time. It went very fast. And basically was run down of all the developer principles, acronyms, etc. A good talk. But i expected more from the father of PHPUnit.

Advanced OO patterns by Tobias Schlitt

The best session of the day. Tobias is a real speaker. One that say the right things. And makes the audience laugh. I really enjoyed listening to to him talk. But a subject like Advanced OO patterns can only be cool. So i guess i he chose the right talk to give. Like most talkers he gave some props to Martin Fowler and the Gang of Four. For writing on patterns and refactoring. And basically it’s a refreshment of studying design patterns. Which i do an a daily basis.

There was one real gem however in this session that i hadn’t come across yet. Which basically is a way to do lazy loading with objects. This is nothing new. But the framework he used for the talk. Used the new 5.3 closures for this. And i really liked that. So besides the fact that i learned a cool pattern. I also learned a new way of using closures in PHP 5.3.

A great conference

That basically was the conference for me. I really was looking forward to listing in on Derick’s talk about profiling PHP applications. But due to my condition i had to stay home the last day and miss this talk. A bit sad. Thankfully he has pretty detailed PDF up on his site about the talk.

Once again IBuildings did a great job in organizing this event. Thanks guys. You once again did an awesome job. I do have some small notes however. During the sessions i heard a lot of speakers say. This is a sessions for 60 minutes. And i only have 45 minutes. So i have to skip some stuff. I mean come on how long was it known that this event was going to happen. Either give the speakers more time. Or speakers adjust your talks.

Besides the fact that i sometimes had the feeling i was visiting the IBuildings headquarters. It was a great event. Hope to see you there next year.

So lenss.nl was offline Thursday for about 5 hours. When i was getting ready for the DPC11 tutorial day i got a notification that one of my servers was down. So i quickly setup a ssh connection. And that still seemed to work. The only thing off was. Apache wasn’t running anymore. So tried a restart. And was unpleasantly surprised by the result. Which looked something like this.

/usr/sbin/apache2ctl line 82 (core dumped) $HTTPD ${APACHE_ARGUMENTS} -k $ARGV

Googling didn’t provide any useful information about the issue. So i contacted support and asked them to take a look at it. So i could move in the direction of the DPC11. After some horrible traffic and a closed road due to an accident. I finally made it to the DPC 2 hours late. But when i plugged my laptop in i noticed apache was still not running. Contacted support again. They couldn’t find any serious issues. I already missed the first part of “Optimizing MySQL Essentials“. So decided to do the tutorial first.

While i was sitting there i got the idea of disabling the apache modules. Which turned out to be a good idea. So after the tutorial session finished i logged in to the server. Disabled all apache modules and got apache working again. After some testing the cause of the problem seemed to be the Google mod_pagespeed apache module. That’s the second time this happened after a upgrade. Will leave it disabled for now.

On a side note.. Where is all the direct traffic coming from all of the sudden?

It was a long time ago i placed a blog entry. Due to work etc i didn’t had any time for it. But now a new entry from me for the delphi developers under us.

Within delphi 2010 the RTTY RTTI system has been completely renewed allowing us delphi developers to do more with RTTY RTTI. One of the things i do with RTTY RTTI is JSON serialization which is a new feature within delphi 2010. For a quick sample regarding JSON in delphi 2010 i refer to the following site: http://blogs.embarcadero.com/adrian/2009/08/19/json-types-for-server-methods-in-datasnap-2010/

Another new feature in delphi 2010 is custom attributes. With custom attributes you can define some extra information on a property like c#. For example i have a object called foo. This object has a published property called doo. This property i want to give some extra attributes for example a fieldname which will be used in the database for updating.  To define such attribute and use it on top of a property i’ll describe in the code snippet below.

//First define the custom attribute(s) you want to have. Within these attributes you can do several
//things. even validation of the field is possible.
//
type TMyAttribute = class(TCustomAttribute)
private
FFieldName : string;
FFieldWidth : Integer;
public
constructor Create(const FieldName : String; FieldWidth: Integer);
property Fieldname : String read FFieldname write FFieldName;
property FieldWidth : Integer read FFieldWidth write FFieldWidth;
end;
//Now we have a attribute defined we create a object which will use that attribute on a property.
//If you take a look at the property Field you notice it on top of that property there is a [TMyAttribute] tag.
//This is how you add a custom attribute on top of a property.
//I also added a property Data without a custom attribute to show the difference.
//
type TFoo = class(TObject)
private
FField : string;
FData: string;
published
[TMyAttribute('VELD1', 60)]
property Field : String read FField write FField;
property Data : string read FData write FData;
end;
type TRttiTest = class(TObject)
public
procedure rttiDemo;
end;
constructor TMyAttribute.Create(const FieldName : String; fieldWidth : integer);
begin
FFieldName := Fieldname;
FFieldWidth := FieldWidth;
end;
//here we are going use RTTY RTTI to go through the properties, check/read the attributes on the property and
//Take some action on it.
//
procedure TRttiTest.rttiDemo;
var
oFoo : TFoo;
oRttiType : TRttiType;
oRttiContext : TRttiContext;
oAttrib : TCustomAttribute;
fFieldName : String;
fFieldWidth : integer;
begin
try
oFoo := TFoo.Create;
oRttiContext := TRttiContext.Create;
oRttiType := oRttiContext.GetType(oFoo.ClassType);
for oAttrib in oRttiType.GetAttributes do
begin
if oAttrib is TMyAttribute then
begin
fFieldName := TMyAttribute(oAttrib).FieldName;
fFieldWidth := TMyAttribute(oAttrib).FieldWidth;
...
end;
end;
finally
FreeAndNil(oFoo);
FreeAndNil(oRttiContext);
end;
end;

As you can see custom attributes can be quite handy. I used it within a application that creates and maintains the database behind it based on the object structure. When a new property is being defined the application automatically creates the new field into the database for me when saving the contents of that object. This system gives me more time for development purposes instead of database management etc. Off course this was my choice of using the attributes system.