knightly

Blog Archives

PHP & DOMDocument slow loading pages

While trying to load a very basic XML template with DOMDocument::load() i ran into some issues last Friday. During the weekend i completely forgot about this. Thank god :) But since it’s Monday i have no other option then to solve it. The problem i was experiencing was page loads took about 2 minutes for every controller action in the application. My first thought was that it would be related to permissions. The application not being able to write certain files. But after checking all permissions the problems still remained.

The logs weren’t showing anything. No errors, nothing. Time to enable the XDebug profiler. And after doing a few requests and loading them up in KCachegrind i found the root of the problem quite fast. A call to the mkDom method was causing the slow load.

But because there are no errors or other indications something is wrong. I added some debug statements to the code in question. And found out that it was PHP’s native DOMDocument::load() that was taking two minutes to load a 200k XML template. That’s not good. So what’s going on here?

First let’s check why there are no errors. Turns out that for some specific reason LIBXML_NOERROR is set. And after disabling this i was presented with an error.

Caught exception: ErrorException: DOMDocument::load()
Entity ‘nbsp’ not defined in /some/file.tal.html, line: 363

WTF? HTML entities inside a XML template. O well. So i changed the  ’s inside the template to their XML equivalent  

Reloaded the page and everything is fast again. This solved my problem. But i wasn’t happy yet. Nobody else was experiencing this issue in the office. So there had to be some other issue. That’s when Geoff notified me about the fact that DTD checks are disabled by using LIBXML_NONET. For this to work however there needs to be an extra W3C package installed.

This was one of those hard to track errors. But it’s fixed and everything is running smooth again.

Writing a simple WordPress plugin

In the spirit of Ideas for March i should really be blogging more. Somehow i just can’t think of anything interesting at the moment. Will give it a shot anyway.

Last week i created my first ever WordPress plugin. It’s an extremely simple one. And for sure nothing new or special. But it was a fun experience. I have always dreaded the WP code base. The mixture of OO and procedural programming just looks plain ugly to me. But i don’t want to be bash WP. I use it myself. So i took the dive and spend a few hours hacking and reading. And the result is my DWOTD (Dutch Word Of The Day) plugin.

To get some inspiration for this i took a look at some plugin code available on the internet. This gave me a basic understanding of how the plugin system works. Together with the docs writing a plugin wasn’t that hard. The code that i saw however was pretty bad. And i wanted something that at least looked better then all the global statements swarming around. So i created a small OO wrapper for the plugin. The basic plugin structure looks something like this

– plugin
—– css
———- dwotd.css
—– lib
———- Dwotd.php
———- Widget.php
—– template
———- Widget.php
– dwotd.php

The file in the root is the bootstrap file. It includes the main plugin class and registers the widget class. The two files in lib contain the plugin and Widget classes. The rest is self explanatory i think.

The first lines of the bootstrap file contain a comment block that is used by WP to show some more information about the plugin inside the admin.

/*
Plugin Name: DWOTD
Plugin URI: http://lenss.nl/projects/dwotd-plugin/
Description: Dutch Word Of The Day including English translation
Author: Thijs Lensselink
Author URI: http://lenss.nl/
Version: 1.2
*/

Then the plugin class file is loaded and the a plugin object instantiated.

require_once(dirname(__FILE__).'/lib/Dwotd.php');

$DWOTD = new DWOTD_Plugin();
wp_register_style('dwotd', "{$DWOTD->getPluginDir()}/css/dwotd.css");
wp_enqueue_style('dwotd', "{$DWOTD->getPluginDir()}/css/dwotd.css");

If the logged in user is a admin hooks for installing and uninstalling the plugin will be activated.

if ( is_admin() ) {
	register_activation_hook(__FILE__, array($DWOTD, 'install'));
	register_deactivation_hook(__FILE__, array($DWOTD, 'uninstall'));
}

And finally register the Widget class

add_action('widgets_init', create_function('', 'return register_widget("DWOTD_Widget");'));

The Widget class is a simple wrapper around the plugin class. It basically instantiates the Plugin class and after that the widget() method gets called. This method calls the Plugin and finally renders a view for the widget box.

class DWOTD_Widget extends WP_Widget
{
	protected $_dwotd;

	public function DWOTD_Widget()
	{
		$this->_dwotd = new DWOTD_Plugin();
		parent::WP_Widget(false, $name = 'DWOTD Plugin');
	}

	public function widget($args, $instance)
	{
		$result = $this->_dwotd->getFromCache();
		if (!$result || $result->used_on != date('Y-m-d')) {
			try {
				$result = $this->_dwotd->fetchRandomWord();
				$this->_dwotd->writeToCache($result);
			} catch (Exception $e) {
				return new WP_Error('broke', __($e->getMessage()));
			}
		}

		$this->_renderView($result);
	}

	protected function _renderView($data)
	{
		ob_start();
		include $this->_dwotd->getTemplate();
		ob_get_contents();
	}

	public function update($new_instance, $old_instance)
	{
		$instance = $old_instance;
		return $instance;
	}
}

All real work is done inside the the Plugin class. This has all methods for talking to the database instance. But also contains the hooks for the WP install and uninstall actions. Some of the contents of this method are listed below.

require_once(ABSPATH.'wp-admin/includes/upgrade.php');
require_once(dirname(__FILE__).'/Widget.php');

Class DWOTD_Plugin
{
	public function __construct()
	{
		global $wpdb, $table_prefix;
                // set some class vars and include some files
	}

	public function install()
	{
                // create database and insert data
		add_option('dwotd_quote','1');
	}	

	public function uninstall()
	{
		$this->_dbh->query("DROP TABLE IF EXISTS `{$this->_dbName}`");
		delete_option('dwotd_quote');
	}

	public function getPluginDir()
	{
		return $this->_wpContent.'/plugins/dwotd';
	}
}

The code is just a small part of the real code. Which you can download here. After that it’s quite easy. Create a folder for the plugin in wp-content/plugins and copy the files there. Enable it from the plugin manager and enable it in the widget menu (which didn’t work for me because my theme has no sidebar.. well not the wp one :)). So i had to use a manual approach.

After copying the files to the plugin folder login to wp-admin and browse to Plugins/Plugins. Search for the plugin and click activate.

Enabling the widget from the admin panel is easy. Browse to Appearance/Widgets. And drag the widget from the left to the sidebar panel on the right.


Enabling it the manual way is also easy. But required a bit more work. For this to work the correct theme file needs to be edited somewhere in wp-content/themes. And the following code needs to be added in the spot the widget should appear.

$args = array ( 'title' => '' );
$instance = array ( 'title' => 'DWOTD', 'number' => 1 );
$widget = new DWOTD_Widget();
$widget->widget($args,$instance);

That’s all. The result can be viewed on the right top side. I build the plugin for fun and use on an Intranet. So don’t go blaming me if it breaks something. Besides that everybody is free to do with the code as they please.

PHP’s current trend status

Kevin Schroeder has a nice post about the current state of PHP in the TIOBE index. And i have seen this TIOBE index pop up every now and then in blog posts. It never really interested me. I am not a big fan of trend statistics and certainly don’t care for what is the hottest language of the moment. But reading Kevin’s article got me interested. He mentions the fact that if you do manual searches on different search engine with a query like +”PHP programming” vs +”Python programming” you will get a different view then what the TIOBE index is displaying. And this is true. So let’s give that a try.

+”PHP programming”
Google : 1.460.000
Yahoo : 132,000,000
Bing : 133.000.000
Baidu : 100,000,000

Let’s do the same for Python.

+”Python programming”
Google : 952.000
Yahoo : 22,700,000
Bing : 27.200.000
Baidu : 1,860,000

But this only displays the fact that there is way more content about PHP on the web then there is about Python. It does not reflect the current request and interest in the two subjects. Plus i know some Python programmers. And i also know for a fact that the demand for Python developers has been growing slowly. More conferences are popping up. So the community is live and kicking. So how about the PHP community? There is still a steady flow of fresh content, blog posts, new and old conferences, activity in php.general (dropped a little), new versions are delivered, php.internals is busy as always and the demand for PHP developers is at an all time high. At least here it is. So this community is live and kicking as well.

So this got me reading the TIOBE Programming Community Index Definition to see if i could figure out how they calculate the final result. I am by no means a mathematics person. But the formula seems straight forward. What happens inside the hits() and hits50() function is not clear however. So matching their findings against mine is not really possible.

((hits(PL,SE1)/hits50(SE1) + … + hits(PL,SEn)/hits50(SEn))/n

So how else can we see the current state of PHP then. Let’s try Google trends. That’s what it’s there for.

Compare search requests for “PHP” vs “Python”

Interesting results. This shows that the interest for PHP has been steadily dropping since somewhere around 2004. When people started getting scared of PHP5 i assume. Python on the other hand is nearly visible but steady.

Let’s try a more precise query “PHP programming” vs “Python programming”

That will display something i did not hope for. But something i was expecting non the less. The interest for Python programming has risen over the past years.

One more comparison for the road “PHP developer” vs “Python developer”

Well that’s kind of a relieve. It shows the demand for Python developers is growing. But they have a long way to go to catch up on PHP :)

Conclusion for me is. I still have no clue what the TIOBE index does. Google trend shows a drop in PHP’s popularity and a rise in Pythons Popularity. Personally i don’t see much change in the PHP community compared to resent years. And i have no real opinion of what is going on inside the Python community. But i am happy to stick with PHP anyway.

Apache SSL client side certificate data in PHP

What should have been a simple assignment turned out to be a hair pulling endeavour. The ultimate goal was to read the client side certificate data in PHP. I am by no means a system administrator. And the SSL part will probably be done by somebody more experienced. And the certificates will be signed by real CA’s. But for developing locally i need something functioning.

So i spend the last hours trying to get client side certificates working. With absolutely no luck. I found a bunch of posts by doing Google searches. But none of them seem to offer the proper information for creating good client side certificates. Creating the CA and the server certificate is no problem at all. But creating a client side certificate seems impossible. Some of the post i tried:

http://www.modssl.org/docs/2.7/ssl_faq.html
http://www.vanemery.com/Linux/Apache/apache-SSL.html
http://www.linuxconfig.org/apache-web-server-ssl-authentication
https://help.ubuntu.com/community/OpenSSL
https://help.ubuntu.com/8.04/serverguide/C/httpd.html
http://it.toolbox.com/blogs/securitymonkey/howto-securing-a-website-with-client-ssl-certificates-11500

You would have thought that something like this would have been documented pretty well by now. But no luck for me. This only resulted in

[debug] ssl_engine_kernel.c(1879): OpenSSL: Read: SSLv3 read client certificate A
[debug] ssl_engine_kernel.c(1898): OpenSSL: Exit: failed in SSLv3 read client certificate A
[info] [client xxx.xxx.xxx.xx] SSL library error 1 in handshake (server lab:443)
[info] SSL Library Error: 336151570 error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate Subject CN in certificate not server name or identical to CA!?

So after almost giving up i found the CA.sh script hidden in /usr/lib/ssl/misc this little sucker seems to do the job pretty well. Creating a CA, server certificate and client side certificate is extremely easy. So i settled for that.

Creating the CA

$ cd /usr/lib/ssl/misc
$ /CA.sh -newca
CA certificate filename (or enter to create)

Making CA certificate …
Generating a 1024 bit RSA private key
…………………..++++++
………………………………++++++
writing new private key to ‘./demoCA/private/./cakey.pem’
Enter PEM pass phrase:
Verifying – Enter PEM pass phrase:

And fill out some basic certificate data

Country Name (2 letter code) [AU]:NL
State or Province Name (full name) [Some-State]:NH
Locality Name (eg, city) []:Purmerend
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Bluesignal
Organizational Unit Name (eg, section) []:lab
Common Name (eg, YOUR name) []:lab
Email Address []:my@email.tld

Creating the server certificates

$ ./CA.sh -newreq
Generating a 1024 bit RSA private key
.++++++
…………………………………………………………………………++++++
writing new private key to ‘newkey.pem’
Enter PEM pass phrase:
Verifying – Enter PEM pass phrase:

Fill out the same basic certificate data

Country Name (2 letter code) [AU]:NL
State or Province Name (full name) [Some-State]:NH
Locality Name (eg, city) []:Purmerend
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Bluesignal
Organizational Unit Name (eg, section) []:lab
Common Name (eg, YOUR name) []:lab
Email Address []:my@email.tld

Sign the sucker

$ ./CA.sh -sign
Using configuration from /etc/ssl/openssl.cnf
Enter pass phrase for ./demoCA/private/cakey.pem:
Check that the request matches the signature
Signature ok

The only thing left to do is creating the client side certificate

openssl pkcs12 -export -in newcert.pem -inkey newkey.key -out username.p12 -name “Client Certificate”

Time to configure Apache2. I used the standard default-ssl virtual host and just reconfigured it

SSLEngine on
SSLProtocol -all +TLSv1 +SSLv3
SSLCipherSuite HIGH:MEDIUM
SSLProxyEngine off
SSLOptions +StrictRequire +OptRenegotiate +StdEnvVars +ExportCertData

SSLCertificateFile /usr/lib/ssl/misc/newcert.pem
SSLCertificateKeyFile /usr/lib/ssl/misc/newkey.pem
SSLVerifyClient require
SSLVerifyDepth 1

SSLCertificateChainFile /usr/lib/ssl/misc/demoCA/cacert.pem
SSLCACertificatePath /usr/lib/ssl/misc/demoCA/certs
SSLCACertificateFile /usr/lib/ssl/misc/demoCA/cacert.pem

reboot Apache2

$ /etc/init.d/apache2 restart

The server side is ready. But it is still impossible to connect at this moment. We need to install the client certificate inside Firefox

Edit > Preferences > Advanced > View Certificates

Choose import and browse to the newly created *.p12 certificate file.

Now i can finally connect based on my client side certificate and read the pieces of data i was looking for. Which can easily found by doing

print_r($_SERVER);

Some of the stuff i was looking for

[SSL_CLIENT_S_DN_C] => NL
[SSL_CLIENT_S_DN_ST] => NH
[SSL_CLIENT_S_DN_L] => Purmerend
[SSL_CLIENT_S_DN_O] => Bluesignal
[SSL_CLIENT_S_DN_OU] => lab
[SSL_CLIENT_S_DN_CN] => lab
[SSL_CLIENT_S_DN_Email] => my@email.tld
[SSL_CLIENT_I_DN_C] => NL
[SSL_CLIENT_I_DN_ST] => NH
[SSL_CLIENT_I_DN_O] => Bluesignal
[SSL_CLIENT_I_DN_OU] => lab
[SSL_CLIENT_I_DN_CN] => lab
[SSL_CLIENT_I_DN_Email] => my@email.tld

Now it’s time for the fun part.

Voices of the ElePHPant

Just found this nice little project Cal Evans is doing. It’s called Voices of the ElePHPant. The name i guess is dedicated to Vincent Pontier the creator of the elePHPant. The goal is to shine a light on a special PHP community person. And the kick of is done by Jeremy Kendall. Be sure to check back often for new updates. Or nominate somebody.

You want to do what with PHP?

I finally took the time to read ‘You want to do what with PHP?‘. Normally i would not write about the books i read. But this book is a bit different. And since i won a copy of Kevin Schroeder’s book in a twitter sweepstake. I thought i might as well write a small review to show a bit of gratitude. I will keep it short.

Kevin is Zend Evangelist for products like the Zend Framework, Zend Server, Zend Studio and co writer of the book The IBM i Programmer’s Guide to PHP

It’s been a while since i read a PHP related book. I have been doing PHP based development for the past 10 years. And after reading almost every book of value on this topic. It’s hard to find something interesting to read about it. Most books just go over the basics. Or target a specific project / way of developing.

This book is a bit different from what i have read so far. And it will get a nice place near my other books of value. So what does make this book so different? First of all the topics that pass by are some of the more advanced topics you will come by in the PHP world. A lot of it is related to low level programming. So a bit of experience or interest in this subject is a must. Besides the advanced topics Kevin shows how to solve issues in ways i have never done them before. And that probably comes down to his experience with more low level languages like C.

He touches topics like Networking and sockets, Binary protocols, character encoding, streams, SPL, Asynchronous operations, file access, Daemons, And two great chapters about debugging, good development practises and just how to become a better developer in general. The book has a considerable amount of binary math in it which is cool. And which is also needed when doing things like handling raw TCP/IP and TCP/UDP data, writing stream handlers or creating your own file-system.

Maybe not all of the material touched in the book is relevant to web development. And most of us will probably never have to write an HTTP daemon in PHP. But the way Kevin tackles problems showed me things i never thought of before and new ways to attack old problems.

It gave me some good pointers for a webcrawler i have been working on. As i have wanted to add threading to this for a long time now. Besides that i just had a lot of fun reading this book. And would definitely advocate other developers to read it as well.

Added a review on Amazon as well.

New generation elePHPants arrived!!

Yesterday i got a message that the new generation of elePHPants are available for bulk pre-order. I am very happy the elephpant.com site is finally up and running. For the past two years i have been shipping a lot of these blue guys all across the world. No need for me to do that any more.

A big thanks for this furry blue creature goes out to:

Vincent Pontier
Damien Seguy
Christophe Villeneuve
Cesar Rodas

Compiling PHP v1.0.8

Today Rasmus posted on twitter that he just compiled PHP v 1.0.8. And it actually worked. Is that great? :) He was in need of this for his upcoming talk this week in Paris. But i couldn’t resist and try to compile this ancient piece of work ;)

$ wget http://museum.php.net/php1/php-108.tar.gz
$ tar xfvz php-108.tar.gz
$ cd php

If you need to change any config setting do it now. And hit make!

$ vi config.h
$ make

This creates a few .cgi files. So it actually does still build

phpf.cgi
phpl.cgi
phplmon.cgi
phplview.cgi

Let’s copy that to my cgi-bin and run it through Apache.

Jquery & PHP doing simple image slicing

Today i was in need of a basic image slicer. An there must be a million of these things out there. But i wanted to see how hard / easy it would be to create this myself.

Conclusion. It’s not that hard when using the jquery library. which is becoming my Javascript framework of choice.



I started with a div.

Then i added the placeholder for the to be scaled image. And a div that will represent the slicer borders.

The next thing to do is setup the scaling canvas. So let’s make it draggable and resizeable.

// Create a draggable / scalable slicer
       $(function() {
            $('#image_slicer')
            .draggable({containment:'#image_slicer_canvas'}) // constraint
            .resizable();

            // set the slicer canvas size
            resizeCanvas(
            	$('#image_slicer_image').width(),
            	$('#image_slicer_image').height()
            );
       });

The only thing we need now is a bit of javascript to handle the slice action. I used a sample form to do the posting to the slicer.php script

// handle slicer actions
       $('#image_slicer').resizable({
   	      stop: function(event, ui) {
   	          var pos = $('#image_slicer').position();
   	          // width, height, left, top
   	          $('#scaler_width').val($('#image_slicer').width());
      	      $('#scaler_height').val($('#image_slicer').height());
        	  $('#scaler_left').val(pos.left);
        	  $('#scaler_top').val(pos.top);
   	      }
       });

The result is a nice red dotted line (square) on top of the image. The dotted area is draggable and resizeable.
When the right slice is selected just hit the slice button and PHP/GD will do the rest.

$width  = $_POST['width'];
$height = $_POST['height'];
$left   = $_POST['left'];
$top    = $_POST['top'];

// read source image
$src = imagecreatefromjpeg('images/sample.jpg');
$dest = imagecreatetruecolor($width, $height);

imagecopy($dest, $src, 0, 0, $left, $top, $width, $height);

header('Content-Type: image/jpeg');
imagegif($dest);

imagedestroy($dest);
imagedestroy($src);

That’s all. All files can be downloaded here

Zend Studio goes virtual

Zend Studio added a new great feature to there already impressive stack of features. The IDE just keeps growing in the right direction.

So the new 8 version which at the time of writing is in Beta added VMWare support. This is a feature i was looking forward to. And couldn’t wait to give this a try. So when i got a email that the new beta is out. I didn’t wait and downloaded the behemoth.

Adding VMWare support to the IDE seems like a logical thing to do. Specially since everything is virtualized now a days. I thought about this a while. But you can use it in different setups. For me it is an extra on my development environment. I’m a big Linux enthusiast. But at the office i work on windows. So now i can develop in my own virtual Linux environment. But it’s also a great solution for running Unit Test on a production VM. Or just to test on a production VM.

My first try was on my OS of choice which is Ubuntu. That however didn’t go to well. As soon as i started the “Run as VMWare application” my IDE would die on me. I quickly gave up on this mainly because i am running Maverick which is not considered stable. And VMWare already had some problems building the kernel modules. If anybody is interested in the crash logs. I still have some.

So back to windows. Windows 7 in particular. I always keep a Windows partition laying around. For games and stuff. So i downloaded the following files.

* Ubuntu-Server
* Zend Studio 8 Beta (trial)
* VMWare 7.1.2 (trial)

Installing VMWare and Zend Studio is a since. Nothing to make note off here. After that was done i installed Ubuntu 10.04 Server edition. And installed Zend Server inside of it. That should be all.. right? So let’s launch the IDE and give it a shot. Well it launched but i got back an error like below.

At first i had no clue what was going on. But i had the feeling there was something wrong with the VMWare client tools. After digging through the Zend Studio manual i found the answer The ‘hgfs’ was not mounted on teh client OS. And this was because the vmware tools did not have enough libraries to build everything. To build the complete VMWare tools on the client OS we need

* gcc
* make
* build-essential
* linux-headers-(current version)

When that’s done it necessary to attach the linux version of the VMWare tools to the virtual CDRom drive. So we can mount it inside the VM.

/usr/lib/vmware/isoimages/linux.iso

mount /dev/cdrom /mnt

Copy the file somewhere and unmount the /mnt or the build will fail. Unpack the VMWare tools and run the perl script inside to install. Once that’s done follow the VMWare instructions. Or reboot the VM.

Now we are all set. Now it’s just a matter of writing code. And running it on the VM. When running an application as a VMWare application the IDE syncs the files with the shared folder on the VM. After that the internal browser is launched to see the result.

The first time you will be asked for the VM to use. Just point it to the VM image created by VMWare.

When no VM is active when launching the application. Zend Studio will trigger VMWare to launch the VM.

Some output in Zend Studio