Casting Weirdness With PHP

When my coworker today asked if he could cast an array to a object. I couldn’t really answer the question. Don’t think i ever done that. So let’s try. right?

$arr = array('foo' => 'baz');
$obj = (object) $arr;

var_dump($obj);

object(stdClass)#1 (1) {
  ["foo"]=>
  string(3) "baz"
}

Ha that’s cool. It actually works. But wait. What happens when we use a numeric index?

$arr = array(0 => 'bar', 'foo' => 'baz');
$obj = (object) $arr;

var_dump($obj);

object(stdClass)#1 (2) {
  [0]=>
  string(3) "bar"
  ["foo"]=>
  string(3) "baz"
}

WTF? We just created $obj->0 which should not be allowed in PHP as far as i know. So let’s make sure i am not mistaking.

$foo = new stdClass();
$foo->0 = 'bar';

PHP Parse error: syntax error, unexpected ‘0’ (T_LNUMBER), expecting identifier (T_STRING) or variable (T_VARIABLE) or ‘{’ or ‘$’ in foo.php on line 5

But casting the array didn’t complain about a thing. Can we access this property? Well! Not calling it directly. At least not that i know of. But looping over the object’s properties does seem to work.

// Parse error
// var_dump($obj->0);

foreach ($obj as $key => $val) {
    var_dump($key);
    var_dump($obj->$key);
}

int(0)
PHP Notice:  Undefined property: stdClass::$0 in foo.php on line 14
NULL
string(3) "foo"
string(3) "baz"

It’s not that i was planning on ever using this. Or advising other people to use it. In the contrary. But i guess it’s not completely useless. But care is required when doing so.

Zend Optimizer+ Integrated Into 5.5

It looks like PHP is going to get another built in performance boost. The plan is to integrate and open source the Zend Optimizer+ component. The fastest Opcode cacher out there. Zend has provided the source. Which is already available on github. So we get the Opcode cacher to boost out applications even more. And the community can extend and built upon the available source code. Win! Win!

PHP 5+ has come a long way performance wise without the Optimizer component. But being able to apply opcode cache out of the box is a great addition. And so far the benchmarks look promising. The Optimizer+ component beats APC hands down in all performed tests.

Read more about Opcode caching and the integration process here.

PHPness Serious WTF

It’s been a while since i last blogged. And i don’t really have anything new to offer. But i would like to comment on some current event. There has been some commotion about a free t shirt that was distributed on the SunshinePHP conference this year.

The first time i saw the shirt i could not make out what was so offending that people would start blogging about it. Somebody had to point that out for me. So what’s the deal here? Apparently the shirt’s slogan was intentionally created with a slight sexual undertone. The idea comes from the enlarge your penis SPAM we all know!

Some people have perceived this as being sexist. And even go so far as to personally attack the people behind the shirt.

Personally i don’t think the shirt is sexist or offensive at all. And don’t really understand all the fuss. Sex has never been an issue for me in the PHP community. I learn just as much from female developers as i do from the male ones. We are all equal. People that feel the need to defend the opposite sex are probably the ones that see a difference. Even so! Why make such a big deal of something so small? I never seen anybody blog about the programming with attitude poster.

I think the PHP community has just lost a bit of it’s shine and coolness. I’ve always been under the impression the PHP community consisted of free thinking people that do what they love. Develop stuff. And that’s exactly what attracted me to it 12 years ago in the first place.

The community has grown quite a lot in that period of time. The thing with every communities is that it brings forth vocal people. Which is a great thing in general. But not so good when people forget their responsibility towards the rest of the community. And that’s exactly what happened the last few days. Personal issues have become public. To me it seems many of you have missed the boat completely. You don’t have to like the shirt. But there is absolutely no reason to bash the initiative in public.

Here are some of the articles the spawned over the last couple days:

PHPness Gate – raising interesting issues

On Public Outrage And Bad Actors

Sexism in the PHP Community

On Sexism/Racism/Any-other-ism and the PHP Community

Sexism and PHP

Hacker news - Sexism and PHP (calevans.com)

Web & PHP sexist tshirt debate – my 2 cents

On Equality, Sexism and an Even Hand

So let’s all just respect one another. And keep the PHP community a friendly place. This kind of nonsense has no positive impact on anybody.

Publish Google Reader Shared Items

A lot of people are using the shared items feature in Google Reader to publish what they are reading on a blog. It’s like having a live blogroll widget on your website. And gives your readers a good impression of what you are actually interested in.

I never bothered adding this to my blog. But i do like to share. So last night i was trying to figure out how to integrate this with my current blog. I was hoping for an easy implementation. But the information is scarce. So i had to do some digging. I did find a couple of Wordpress plugins. But most of them were not updated for at least a year. And i am not putting code like that life on my website.

After some Googling i came across a couple of feed URL’s that seem to share their shared items in feed form. Using the Google Reader URL and a user id. So the URL for my shared items would look like the one below. Getting your user id by the way is easy. Go to your Google reader page. Click all All items and the UID will show up in the address bar.

http://www.google.com/reader/shared/16525759780220726764

The problem with this however. Non of my shared posts show up on this page. And i have not figured out a way to populate it just yet. So i came up with an other path to get this data on my server.

In the Google Reader pages it’s possible to add sharing functionality. This is done by going to the Send To tab

From here it’s possible to select a service to share data with. Non of these services are any good for what i am trying to do. But the great thing about this page is. You can provide custom URL’s for sharing data. Just incorporate the specified parameters in the URL and your done.

Once configured and saved a custom URL is visible in the Send To panel

That’s it for this part. Actually sharing items is quite easy now. When back in the Google Reader under each post there is Send to link. If you click this link the newly created custom URL will show up. And you can share the post.

The only thing left to do is write some code to process the incoming data. To get you started. Some of the code i used while testing this is posted below

Class GoogleReaderShare
{
    const GOOGLE_REMOTE_IP = '95.97.54.3';

    const GOOGLE_REFERER = 'http://www.google.nl/reader/view/';

    protected static $_whitelist = array('source', 'title', 'url', 'short');

    public static function CheckSource($remoteAddr, $referer) 
    {
        if (($remoteAddr == self::GOOGLE_REMOTE_IP) 
                && ($referer == self::GOOGLE_REFERER)) {
            return true;
        }
        return false;
    }

    protected static function _CheckIncomingData($data)
    {
        if (!is_array($data)) {
            return false;
        }

        foreach ($data as $key => $value) {
            if (!in_array($key, self::$_whitelist)) {
                return false;
            }
        }
    }

    public static function Process($data) 
    {
        if (!self::_CheckIncomingData($data)) {
            throw new Exception("Unrecognized or no incoming data");
        }

        // process the data
    }
}

if (GoogleReaderShare::CheckSource($_SERVER['REMOTE_ADDR'], $_SERVER['HTTP_REFERER'])) {
    GoogleReaderShare::Process($_GET);
}

Long Running PHP Script and MySQL Server Gone Away

At the moment i am writing some workers that interact with a beanstalk server. When data gets pushed into the beanstalk server. My workers will be triggered to process this data. So the workers are sitting idle for most of the time. And just wait for some data to process. No rocket science there. But during testing i kept running into MySQL issues. The script seemed to lose connection to the MySQL server when it was idle for longer then a minute. And would respond with a

Warning: mysql_query(): MySQL server has gone away in /path/to/some/mysql4/class.php on line xxx

So whats going on here? Well actually it’s quite simple. When my script initializes the database connection it doesn’t use it. It just sits there waiting for incoming data. Once received it will process it and try to store it in the database. But when the waiting period exceeds the time for PHP to keep the MySQL connection open it responds with the warning mentioned above. Now in previous versions of PHP this would not be a big issue. As PHP would just initiate a reconnect when the connection is lost. But from PHP 5.0.3 and up this functionality has been disabled by default. For MySQLI this is no problem at all.

MySQLi:

mysqli.reconnect = Off to On

Unfortunately i am working with PHP’s core mysql_* functions (don’t get me started) and there doesn’t seem to be an easy way to resolve this. According to the MySQL documentation

mysql_options(&mysql;, MYSQL_OPT_RECONNECT, &reconnect;);

Should do the trick. But passing MYSQL_OPT_RECONNECT in anyway to mysql_connect didn’t result in the result i was looking for. So what now? Porting the database code to make use of the newer and better PDO or MySQLi is no option. As it would consume way to much time. Fortunately the mysql_* core functions come with mysql_ping. I never had to use this before. But in this case it comes in quite handy.

From the PHP manual:

bool mysql_ping ([ resource $link_identifier = NULL ] )

Checks whether or not the connection to the server is working. If it has gone down, an automatic reconnection is attempted. This function can be used by scripts that remain idle for a long while, to check whether or not the server has closed the connection and reconnect if necessary.

Note: Automatic reconnection is disabled by default in versions of MySQL >= 5.0.3.

Adding the mysql_ping function call was rather straight forward. And didn’t require all that much work to be done. I extended the database class to include a ping method. That would simply throw an exception when it failed to reconnect.

public function ping()
{
    if (!mysql_ping($this->db_connect_id)) {
        throw new Mollie_Database_Exception("Connection was lost");
    }   
    return true;
}

And after that i started poking around the userland implementation. The worker is running inside a while(true) loop. A first test with ->ping() being called inside this loop proved to resolve the issue at hand. But running the ping function that often is overkill. And who knows it might actually result in DoS of the database server. So i decided to ping the server every one minute or so.

$start = time();
while (true) 
{
    $this->_keepConnectionAlive($start);

And the keepConnectionAlive method looks something like this

protected function _keepConnectionAlive(&$start)
{
    $passed = (time() - $start);
    if ($passed > 60)
    {
        $start = time();
        $this->_db->ping();
    }
}

I’m not a big fan of this solution. And would rather be implementing MySQLi functions. But this functions well. And will do for now.

Git: Rebase on Pull by Default

I was a bit surprised today when git presented me a merge message window after i did a pull. This should not happen as the normal behavior here should be to rebase the changes. But apparently that didn’t happen this time. When i asked one of the other dev ‘s what could be the issue. We quickly figured out i was just missing some config entry in .git/config. This probably happened some time ago when i did a fresh checkout.

So to make sure rebasing is done by default. You can run a simple git command or modify the .git/config file manually.

$ git config branch.autosetuprebase always

or do

$ vi .git/config

Make sure [branch “master”] has rebase set to true. It should look like the snippet below.

[branch “master”]

remote = origin
merge = refs/heads/master
rebase = true

Starting to really like git. A couple of more quirks and things will be running fine.

Workaround Ubuntu 12.04 Mysterious System Freezes

I have two machines running Ubuntu 12.04. One of them is very stable. And hardly ever gets rebooted. The other machine is displaying some odd behavior every now and then. And with odd behavior i mean. It just completely freezing up. The only thing functioning at that moment is the mouse.

I use this machine daily. And i can’t afford to lose work due to system crashes. So i could spend numerous hours trying to figure out what’s going on. But that’s probably something better left for the Ubuntu devs them selfs. Besides that there are plenty of bug reports floating around that describe this behavior. And one of those posts seemed to resolve the issue for me.

Apparently the bug that causes these crashes is fixed in 12.10. But the changes will only be backported when 12.10 is released. So that leaves me in quite a pickle. But according to the thread. Upgrading the kernel should do the trick. So that’s exactly what i did. The kernel packages i used can be found here http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.4-precise/

$ wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.4-precise/linux-headers-3.4.0-030400_3.4.0-030400.201205210521_all.deb $ sudo dpkg -i linux-headers-3.4.0-030400_3.4.0-030400.201205210521_all.deb

$ wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.4-precise/linux-headers-3.4.0-030400-generic_3.4.0-030400.201205210521_amd64.deb $ sudo dpkg -i linux-headers-3.4.0-030400-generic_3.4.0-030400.201205210521_amd64.deb

$ wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.4-precise/linux-image-3.4.0-030400-generic_3.4.0-030400.201205210521_amd64.deb $ sudo dpkg -i linux-image-3.4.0-030400-generic_3.4.0-030400.201205210521_amd64.deb

This comes with a downside of course. All modules compiled for the current kernel need to be recompiled. And i haven’t figured out how to rebuild all of them at once. So i just ran the command below for VirtualBox and the NVidia drivers.

$ sudo dpkg-reconfigure package-name

It’s probably a better idea to keep the stable kernel for now. But if system crashes are really bugging you. Then this might resolve the issue. Just be careful.

Fixing Character Replacement in Wordpress

This has been bugging me for a while. But not enough to actually look into it. And Google searches for Wordpress display wrong characters results in a whole forest of threads about UTF-8 encoding. This has nothing to do with that. So what’s the issue here?

Wordpress is replacing characters in my posts. The most annoying being the replacement of – is with - . Which makes no sense at all. Specially when creating code samples. Or command line parameters. My thought is why on earth would you do that? But it probably has something to do with typography…

Anyways. Searching for it today made me stumble onto this post from 2007 by Paul Betts. And believe it or not. Wordpress is still doing that. The code has changed slightly tough. So i created a quick patch as a temporary solution. And need to figure out if this can be circumvented by a plugin or something. If that doesn’t exists already :)

The patch will replace the following lines

$static_characters = array_merge( array( '---', ' -- ', '--', ' - ', 'xn-', '...', '``', '\'\'', ' (tm)' ), $cockney );
$static_replacements = array_merge( array( $em_dash, ' ' . $em_dash . ' ', $en_dash, ' ' . $en_dash . ' ', 'xn--', '…', $opening_quote, 
    $closing_quote, ' ™' ), $cockneyreplace 
);

With

$static_characters = $cockney;
$static_replacements = $cockneyreplace;

Patching the file is easy.

This patch was created for v3.4.2. And will most likely fail on other versions of Wordpress.

$ cd wp-includes/ $ wget https://raw.github.com/tlenss/misc/master/patch/wordpress-charreplace.patch $ patch -p0 < wordpress-charreplace.patch patching file formatting.php

That’s all.

Resize a Vagrant VMDK Drive

Using Vagrant as a development environment is great. But you have to watch the available disk space when doing some funky database benchmarking. Last Friday i wrote a script to populate a test database with over 60 million records. And left it running over the weekend. With the idea it would be finished when i returned to the office on Monday. Well it actually was still running. But the Vagrant instance was out of disk space when i logged in. Damn. Should have made sure i had a big disk attached to this instance.

O well. My first idea was to just drop the database that i wanted to use for testing. But due to no space available on the system. I could not drop it. I really couldn’t do anything on the system. Even writing config files failed. So where did all the disk space go?

$ df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/debian-605-root 10G 9.8G 0.2G 99% /

That’s really not that much space left. And /var/mysql/ibdata1 for the InnoDB tables was taking up 7 GB. And since i have a couple of InnoDB tables i can’t just delete this file. So need to come up with something else. And after a bit of Googling. I came across this post. Which seemed promising. But the most important part is left out. And instead linked to some horrible garbled page. That is completely unreadable. So i had to do some digging to figure this out on my own. For cloning the disks i followed the steps in the article however. The Virtual Box files are located in the user folder in the following path

$ cd ~/VirtualBox VMs/development_*

The first thing i did was convert the VMDK image to a VDI) image so we can actually do some partition resizing.

$ VBoxManage clonehd box-disk1.vmdk box-disk1.vdi –format VDI

So now we have a new VDI image. And we need to resize it so it has sufficient space for what we are trying to do. In my case 20 GB is sufficient.

$ VBoxManage modifyhd box-disk1.vdi –resize 20000

Now that the new disk is created it needs to be attached to the vagrant virtual machine. This is easily achieved by doing right click on the VM in VirtualBox and select Settings

Storage > SATA Controller > Add hard disk (select existing and point to the new VDI image)

Now boot the Vagrant image in VirtualBox so we have a screen to work with. When it has booted up. Login.

First create a new Primary partition in the unused space of type Linux LVM (8e) with cfdisk.

$ cfdisk /dev/sda

When the partition table has been written. It’s time to create a new physical volume on the new partition with the help of pvcreate

$ pvcreate /dev/sda3 successfully created

For the next steps we will be needing the Volume Group name. We will get this by running pvdisplay

$ pvdisplay | grep “VG Name” VG Name debian-605

So now we have all the bits ready. And its time to extend the root partition with the newly created partition.

$ vgextend debian-605 /dev/sda3 Volume group “debian-605” successfully extended

Awesome! next step extending the logical partition

$ lvextend /dev/debian-605/root /dev/sda3 Extending logical volume root to 18.84 GiB Logical volume root successfully resized

Almost there. The last thing left to do is resize the filesystem to make all space available.

$ resize2fs /dev/debian-605/root resize2fs 1.41.12 (17-May-2010) Filesystem at /dev/debian-605/root is mounted on /; on-line resizing required old desc_blocks = 1, new_desc_blocks = 2 Performing an on-line resize of /dev/debian-605/root to 4937728 (4k) blocks. The filesystem on /dev/debian-605/root is now 4937728 blocks long.

It’s that simple really. So let’s check.

$ df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/debian-605-root 19G 8.6G 9.1G 49% /

After this i did not bother to convert the VDI image back to a VDMK image. But if you have to for some reason. You could do that with the commands below. And some manual editing of the VirtualBox xml config.

$ VBoxManage internalcommands sethduuid box-disk1.vdi $ VBoxManage clonehd box-disk1.vdi box-disk1.vmdk –format VMDK

So that’s it really for resizing the Vagrant hard drive. The only thing left for me was to cleanup the large ibdata1 file. And setup Mysql to use separate ibdata files for InnoDB table.

$ vi /etc/mysql/my.cnf [mysqld] innodb_file_per_table

Restart Mysql and dump all needed databases except the system tables like (mysql, information_schema).

$ /etc/init.d/mysql restart $ echo ‘show databases;’ | mysql -uroot -p | grep -v ^Database$ | grep -v ^information_schema$ | grep -v ^mysql$ | xargs mysqldump -uroot -p –databases > dump.sql

When that’s done. Drop all InnoDB database tables. Remove the ibdata and in_log files from /var/lib/mysql.

$ /etc/init.d/mysql stop $ rm /var/mysql/ibdata1 /var/mysql/ib_log

Now we restart Mysql. And reimport the databases.

$ /etc/init.d/mysql start $ mysql -uroot -p < dump.sql

Some of the resources i used for solving this issue:

[1] http://www.turnkeylinux.org/blog/extending-lvm

[2] http://unix.stackexchange.com/questions/37900/gparted-live-cd-cant-resize-a-partition

[3] http://www.my-guides.net/en/guides/general-software/122-how-to-resize-a-virtualbox-disk-partition

[4] http://www.virtualbox.org/manual/ch08.html#vboxmanage-modifyvdi

[5] http://stackoverflow.com/questions/3456159/how-to-shrink-purge-ibdata1-file-in-mysql

GIT: Coming Up With a Good Workflow

A new environment to work in always brings new challenges. And this one is no exception. And i will be working with git from now on. I tried switching before. But always went back to subversion. Because of my job. And for somebody who worked with subversion solely for the past… say 5 to 7 years it’s kind of a new concept and takes some time to get a grasp on. Ok ok. It’s just another version control system. But it takes some getting used to.

And two days ago i kinda got stuck in rebase hell. And had to take some time to figure this all out without doing any damage. Which resulted in another problem. There simply is no revert command like in subversion. But git is pretty smart about this kinda things. So getting the branch full of failed merges back to it’s original state can easily be achieved with two commands and some concentration.

First check the reflog for a point where to start from

$ git reflog

From the output pick the point where to revert back to. Which looks something like HEAD{2}. And run

$ git reset –hard HEAD{2}

So back to square one. My main problem was that i had not kept my branch up to date with the remote repository. Which you can do with the pull command while on the master branch. Or with the rebase command while working on a local branch. Rebase is git’s alternative way of bringing changes together. So the idea behind it is that when your local branch is up to date. The final merge will become as straight forward as possible.

I finally managed to get my changes merged in to master. But did not feel very confident about the whole process. So I need to come up with a steady work flow. And stick to it. To not waste time and run into these rebase disasters.

This is what i have so far. And seems to be a solid routine. Start with a nice new checkout of the project at hand. And branch it for the changes to be done.

$ git clone git://some.repository.url:repos.git $ git branch dev-branch

After working on the newly created branch for a while. Some changes are bound to be ready to be pulled into master. Keeping an eye on the build logs or commit messages should work fine. When this happens it’s time to bring those changes into the dev-branch. After pulling them into master.

$ git checkout master $ git pull $ git checkout dev-branch

If there is a need to rebase the branch with for instance master. This is the moment. This might cause non-fast-forward issues. But i have not completely figured out why i run into these issues every now and then. A way to solve this might be to use the –force switch when pushing to remote. But i am not a big fan of forcing things like this.

$ git rebase

If everything is fine some more work can be done on the dev branch. Until a new change in master is ready to be pulled in. In which case it’s rinse and repeat the previous part. When the changeset is ready to be merged back into master. Switch to master bring it up up o date with a pull. And merge the changes from the dev-branch.

$ git checkout master $ git pull $ git push dev-branch

Again when all is fine all changes will be available in master. If something goes wrong during git-rebase and / or merging. Git will kindly state so. And some manual merging with git’s mergetool will have to be done. But that’s something for another day.

Copyright © 2013 - Thijs Lensselink - Powered by Octopress