f.lux nice tool for the late night / early morning coding sessions http://t.co/q9qP2nM9 2011-09-16
Ok it’s a bit late But i have been laying under a rock for the last week. And i guess it can’t hurt!
Last week a critical bug was discovered in PHP. Which affects versions 5.3.9 and 5.2.17. The bug could be exploited to run arbitrary code on a remote PHP system. So upgrade your systems. And of course Stefan Esser popped up with some wise words :)… O well i still think the guy does great work.
More info about the issue can be found on packetstorm (CVE-2012-0830)
For years PHP has been vulnerable to session adoption which can enable session fixation. And since sessions are a major part of web applications now a days. A lot of platforms are open and waiting for an attack to happen.
session adoption & session fixation
The problem exists because the current session module does not validate the session id that comes in from a cookie. This means uninitialized session id’s can be passed by the client. This happens due to the fact that browsers overwrite cookie if multiple cookies are send per request.
Some people would say this is solvable by implementing session_regenerate_id(). But this is not the case.
Because session fixation can be used to take over control of web applications. Validation is required when multiple cookies are send per request. When multiple cookie are send with a request. Browsers send multiple cookies without domain / path information. This way it’s impossible to tell which cookie belongs to which domain.
So how do we fix this?
There is some userland code that does offer the ability to validate session data. But this has not been widely adopted by other developers.
Code that adds the session ID as a validation key:
session_destory(); session_regenerate_id(); $_SESSION['valid_id'] = session_id();
And the code to check if the session was properly initialized:
if ($_SESSION['valid_id'] !== session_id()) {
die('Invalid use of session ID');
}
Thank god the internal developer know this. And are working to fix this. For the past days there has been an interesting discussion going on on the internals list. About applying a patch that will fix this. The patch will add some new php.ini features and a new method validate_id() for the session save handler. Hopefully this will be available in version 5.4.
To not break BC strict_mode will be disabled by default. But can be enabled by setting the following setting in php.ini. When enabled uninitialized session ID will be discarded.
session.use_strict_mode=0
To prevent a DoS instead of session fixation. An new feature has been added that deletes possible malicious cookies that prevent new session ID.
session.safe_session_cookie=1
You can read more about session fixation and the upcoming patch on the PHP-Wiki
If you’re looking for a new challenging and exiting Senior Webdeveloper position. Don’t look any further. If you already think you have the job of your dreams. Think again!
abbywinters.com (NSFW) is one of the largest and most ethical, highly rated, well designed, and successful erotic websites in the world today. abbywinters.com is the WINNER of the AVN 2011 Awards for Best Membership site!
And we are looking to hire a new talented webdeveloper to expand our small team. What would you think about joining our small Agile team of highly qualified professionals?
You will be creating sexy, exiting and game changing experiences for the web, work for one of the industry leaders. And just be part of an awesome company. Some of the jobs key elements are:
Motivated by principles of social responsibility, we deliver provocative media by embracing imagination, creativity and emerging technologies. Our models, customers and business partners are inspired by our fervid passion.
Our experienced staff use state-of-the-art content production facilities to produce 10 shoots a week from concept to finished art, utilizing the most advanced digital capture, post production and delivery systems in the world.
You will be working directly with our Web Dev Manager, Lead developer and colleagues in the web dev team. We need each individual to contribute for us to continue as a pioneer in our industry.
If you posses a “Can do” attitude. Would like to work in the center of Amsterdam. And are able to identify your self in the criteria below. You might want to head over to our career portal for a more detailed description.
Technical competencies – Required
Technical competencies – Desired
Last week i had a blast while listening to Moxie Marlinspike’s Blackhat talk the past and future of SSL. The video is an absolute must see if you have any concern about the current state of SSL or the whole web for that matter.
As many know. There has been quite the turmoil in the SSL world lately with some big CA’s (Comodo, StartCom, Diginotar, GlobalSign) getting hacked. And rogue certificates being generated for major domains like microsoft.com, google.com, etc for who knows what type of malicious purpose. The real problem here are the centralized CA’s. For SSL to work you need to explicitly trust the major CA’s. And if one gets hacked. Well.. we know the deal by now. And the funny thing is that Moxie mentions in his talk. The whole CA wasn’t even part of the original SSL protocol. As the creator of SSL said “It’s something we through in at the end”.
But instead of only bashing the CA’s. Moxie comes with a solution to the problem. In the form of a new protocol Convergence, Which exists of a client and a server package. The client right now is a simple Firefox extension. And when installed it disables the current CA system in the browser. And will use one or more of the selected notaries instead. It even works with self signed certificates. And the back-end is a modular one. And the standard CA verification can be swapped with for instance DNSSEC based verification.
There is still a level of trust involved. But you won’t lay all your trust in one specific CA. Convergence uses notaries. Notaries are anonymous nodes that can be picked at will and can be used to verify the requested certificate. And like i said the notaries are anonymous. No more browser leakage at this point. One of th notaries will act as a bounce node to which a SSL connection is made. All other notaries are contacted through this secure connection.
I am probably not the best person to explain this all. So go ahead and listen/watch Moxie’s talk and form your own opinion. But i think everybody should install this Firefox plugin. And forget about the whole CA system. I went ahead and installed a notary node myself. Which can be found here. More information about setting up a notary node yourself can be found here
My last post was about the 32-bit integer issue i was having with PHP. And besides setting up a 64-bit server one of the solutions is to compile PHP with the CFLAGS mentioned in my previous post.
Intrigued by the fact that my 32-bit Ubuntu installation did not suffer from the same issue. I started to read the PHP5 changelog for Ubuntu.
CFLAGS=”-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64″ ./configure
But this has a negative side effect. Namely PHP running 50% slower then it should be. And to my surprise Ubuntu applies the CFLAGS above when compiling PHP5 for a 32-bit platform. So that would mean PHP on Ubuntu is always running 50% slower? Well according to this is should be the case.
I downloaded php-5.3.8 from php.net and compiled two versions. One with the CFLAGS set for large files. And one normal without any changes. After that i downloaded the benchmark script from php-benchmark-script.com. And did a couple of runs on each of the two installs. The results are stunning.
PHP-5.3.8 compiled with large file support:
test_math : 4.414 sec.
test_stringmanipulation : 4.968 sec.
test_loops : 3.529 sec.
test_ifelse : 2.344 sec.
————————————–
Total time: : 15.255 sec.
PHP-5.3.8 compiled without large file support:
test_math : 2.274 sec.
test_stringmanipulation : 2.286 sec.
test_loops : 1.619 sec.
test_ifelse : 1.228 sec.
————————————–
Total time: : 7.407 sec.
That’s pretty much a 50% speed decrease. One more thing i tried is adding the ‘AC_SYS_LARGEFILE’ macro to configure.in and rebuild the configure script. But this had no effect at all.
$ cd php-5.3.8
$ vi configure.in (add AC_SYS_LARGEFILE somewhere)
$ export PHP_AUTOCONF=/usr/share/autoconf2.59
$ ./buildconf –force
I then tried two things. First i build without the CFLAGS. But this didn’t seem to do much. Then i used the same configure script with the CFLAGS for large file support. But there was no speed increase measurable.
So why would Ubuntu have made the choice to compile PHP with the large file support on a 32-bit platform? Are there really that much developers that work on large files in web / cli applications written in PHP? Enough to sacrifice a 50% speed decrease?
I don’t really know the answer to that. But i will do my large file processing on 64-bit machines. And will compile PHP from scratch from now on. Until i have upgraded my aging hardware.
So we have a PHP based importer script that does some heavy duty media processing at the office. And i had to import some new media today. But for some reason a couple of files weren’t picked up without a message. So i cleaned up the upload folder. The only files left were the files not being processed. And when i started the importer. The result was.
Importer found (0) files to import!
Hmmm. That’s not right. So i had a look at the code behind the importer. Which basically is a loop using a DirectoryIterator object. And some var_dump calls revealed the issue. For some reason ->isFile() was returning (false) for regular files. WTF! Let’s test that on the command line.
$ php -r “var_dump(is_file(‘/some/file.ext’));”;
bool(false)
Ok so we have an issue here. How big are these files really. A inspection revealed they are all over 2GB. Maybe some 32 bit issue? As the platform the code is running on is a 32 bit server. So i asked my colleagues, Googled a bit and read through php.net. To find out that there is an issue with PHP and files larger then 2GB.
https://bugs.php.net/bug.php?id=27792
https://bugs.php.net/bug.php?id=48886
http://nl.php.net/manual/en/function.filesize.php
Those however all seem related to filesize. The filesize function manual page even has a note about it. Maybe it’s related?
Note: Because PHP’s integer type is signed and many platforms use 32bit integers, filesize() may return unexpected results for files which are larger than 2GB. For files between 2GB and 4GB in size this can usually be overcome by using sprintf(“%u”, filesize($file)).
But i can’t apply that patch on a production server. So i came up with a simple solution for now. I extended the DirectoryIterator class and have overwritten the isFile method. Which works for now (don’t think this will work on windows).
Class MyDirectoryIterator extends DirectoryIterator {
public function isFile() {
return (integer) exec("[ -f {$this->getPathname()} ] && echo 1 || echo 0");
}
}
Convinced it was a 32 bit issue. I came home later that day. And wanted to try it out on my own desktop. That is a 32 bit system and runs Ubuntu 11.04. To my surprise the result was different then i expected.
$ php -r “var_dump(is_file(‘/some/file.ext’));”;
bool(true)
I used the same files as before. And tested some more big files. But the result was the same. Weird. Let’s try some other 32 bit machines.
Ubuntu 11.04: bool(true)
CentOS release 5.6 (Final): bool(false)
Debian 6.0.2 (squeeze): bool(false)
Only my desktop at home seems to have a good result. Ubuntu must have some patch somewhere to fix this issue? To confirm i compiled PHP 5.3.8 from source. And did the same test again on Ubuntu 11.04. And this time it was (false).
$ php -r “var_dump(is_file(‘/some/file.ext’));”;
bool(false)
I am not really in the mood to search the Ubuntu changelog. And for now the work around will do. But i really would like to know what patch is applied to resolve the issue.
[ update ]
While applying the patch for the is_file issue. I was confronted with the fact that way more function calls cause issues. So while waiting for PHP to get patched i had to create some workarounds for the time being.
Getting the filesize:
(integer) exec("stat -c%s {$file->getFilename()}");
Calculate a MD5 checksum:
$md5 = exec("md5sum {$file->getFilename()}");
$expl = explode('\t', $md5);
return (string) $expl[0];
Calculate the CRC32 checksum:
$hash = exec("cksum {$this->path}");
$expl = explode(' ', $hash);
return $expl[0];
Get the modified time:
$stat = explode('.', exec("stat -c%y {$this->path}"));
$timestamp = strtotime($stat[0]);
return $timestamp;
Hopefully that will do for now. On a side note the issue is solvable by setting certain CFLAGS when compiling PHP. I have no idea what the impact of that will be on the PHP binary. But it does seem to solve the issue. Not sure how one would apply that when PHP is installed from the distro’s repository though.
CFLAGS=”-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64″ ./configure
While doing some front end work yesterday. I got trapped by a jQuery issue. Well not JQuery specific. The issue was actually triggered by some other hand crafted code. Every time i would click a link inside my grid view firebug would throw an error.
uncaught exception: Syntax error, unrecognized expression: .
And the markup that triggered the error was
Nothing wrong there right? And it actually took my quite some time to figure this one out. It would be nice to have a tool that can tell you there are multiple click events assigned to a element? But for now it was just some manual searching and testing.
The issue was caused by an other snippet of Javascript code inside another .js file. This piece of code attached a click event to every div inside a grid td. Which may be a bit to greedy.
$('.admin .gridbg tr td span').click(function() {
And my link was in a nested td inside the grid. And also contained a span tag. So it was actually firing off two click events. From which one failed. Fixing it after that was easy. Either make the first click binding less greedy. Or change the markup of my second grid. I choose the last one.
About a week ago i was working on a twitter widget for a website. This required some dates to be displayed in Dutch. And i found out the hard way my knowledge on this has faded away over time.
So the code i was working on. Did something like this.
$date = date('D M d H:i:s Y', strtotime($someVar));
My thought was that by setting the correct locale the dates would appear in the correct language. Wrong!
date_default_timezone_set('Europe/Amsterdam');
setlocale(LC_ALL, 'nl_NL.utf8');
After a reload i was greeted by the same dates as before. In plain English. Oke no worries. Let’s see what setlocale returns.
var_dump(setlocale(LC_ALL, 'nl_NL.utf8'));
bool(false)
That’s not good. Seems like we are missing some locales on the server. Let’s check.
locale -a
en_AG
en_AG.utf8
en_AU.utf8
en_BW.utf8
en_CA.utf8
en_DK.utf8
en_GB.utf8
en_HK.utf8
en_IE.utf8
en_IN
en_IN.utf8
en_NG
en_NG.utf8
en_NZ.utf8
en_PH.utf8
en_SG.utf8
en_US.utf8
en_ZA.utf8
en_ZW.utf8
…
And some more output after that. But not the one i am looking for. But thankfully aptitude was kind enough to provide the missing language packages.
nl_NL
nl_NL@euro
nl_NL.iso88591
nl_NL.iso885915@euro
nl_NL.utf8
So let’s set the correct locale for this script.
setlocale(LC_ALL, 'nl_NL.utf8');
But still no changes. I must be missing something….. Let’s consult the manual. The last line in the examples section is what i was looking for
To format dates in other languages, you should use the setlocale() and strftime() functions instead of date().
Duuh! Completely forgot about strftime. Let’s change the code.
strftime('%a %b %d %H:%M:%S %Y', strtotime($somevar));
ma aug 15 14:55:06 2011
Perfect. That did it.
I was looking for the list of Memcached telnet commands. And couldn’t find much in the docs. So after some Google searching i finally found the list. And will keep it here for future reference.
So i did a bit of reading while working on a memcache tool in PHP. I found some good documentation on the memcached.org website.
| Command | Description | Example |
|---|---|---|
| get | Reads a value | get mykey |
| set | Set a key unconditionally | set mykey 0 60 5 |
| add | Add a new key | add newkey 0 60 5 |
| replace | Overwrite existing key | replace key 0 60 5 |
| append | Append data to existing key | append key 0 60 15 |
| prepend | Prepend data to existing key | prepend key 0 60 15 |
| incr | Increments numerical key value by given number | incr mykey 2 |
| decr | Decrements numerical key value by given number | decr mykey 5 |
| delete | Deletes an existing key | delete mykey |
| flush_all | Invalidate specific items immediately | flush_all |
| Invalidate all items in n seconds | flush_all 900 | |
| stats | Prints general statistics | stats |
| Prints memory statistics | stats slabs | |
| Prints memory statistics | stats malloc | |
| Print higher level allocation statistics | stats items | |
| stats detail | ||
| stats sizes | ||
| Resets statistics | stats reset | |
| version | Prints server version. | version |
| verbosity | Increases log level | verbosity |
| quit | Terminate telnet session | quit |
What a day. Every now and then there are such days where you wished you stayed in bed. Today was one of them. And i was confronted by two weird errors with very non descriptive error message or no error message at all. In both cases the issue was solved without a real solution. This post is just to vent my frustration of the day. And now that i take the time to write this down i might as well add the issue i ran into last week. Will save that one for last.
Reserved keyword
So i have been banging my head over this issue for way to long today. And had to find out the hard way it was a simple little thing as usual. But Doctrine (version 1 if anybody asks) didn’t offer me any good info with the error message it was displaying. Nor did PHP or MySQL (should be MariaDB if you ask me).
While working on a very heavy Doctrine based project. I ran into a at first glance weird error. Yesterday i created a class structure. All good and well. The class i was working on was called Shows. But it represented a single Show entity and therefore should have been called Show. So i decided to do a rename action. Pretty complex because of different dependencies. But after a few tries i got it right. Except Doctrine was failing with a syntax error. After a code inspection i still couldn’t spot the issue. The only thing i didn’t do yet was output the query and run it from a shell. And this failed with the same syntax error message. Damn!
INNER JOIN show s ON …
Checked the whole query. Still couldn’t spot the issue. Then for some weird reason i decided to escape the table name with backticks `Show`. Everything worked again. Wow! So Doctrine doesn’t escape table names. But why did it fail in the first place? By now i had the feeling i was using a reserved keyword. And a check confirmed this. An error would have been helpful here. I had a bit of luck though. As it turned out the table should have been named different anyway. I made a small workaround for it in my code. But decided that was not the way to go So the lesson of the day for me was don’t use reserved words for class / table names. A good thread on the issue can be found here.
APC object corruption
The second issue to bite my ass was a nice one as well. And this time no indication of an error at all. While working on the same classes discusses shortly above. I renamed the class names and the file names on disk. And after a page reload in the browser. The framework was complaining it couldn’t load the changed class. I checked the class and file names and all were correct. After toying with it for 20 minutes i decided to ask my colleague. And he suggested to restart Apache. for the second time today i was stumped. Everything worked again. Turns out that APC couldn’t find the already in memory loaded class. After renaming the corresponding filename (seems logical now that i think of it). A restart of Apache flushed the APC cache and therefore the issue. This sounds like a bug though. And requires some further inspection or a bug report. We will see.
PHP Fatal error
Last week while setting up PHPUnit i ran into a weird issue while testing PHPUnit from a shell on my local desktop. The whole test suite would run from start to end. But at the end would display the following not so descriptive error message.
PHP Fatal error: Exception thrown without a stack frame in Unknown on line 0
This kept me busy for some hours. This happened somewhere in a large framework. And there was no clear sign of what was causing the issue. Some googling did reveal some cases in which this error may occur.
There was an exception while handling an exception
There was an exception while running a destructor
There was an exception while closing the session
There was an exception while running a shutdown function
There was an exception while running a autoload function
The issue in my case was the fact that the configuration file wasn’t loaded yet. And therefore the log file location was not set correctly. Logging in this framework is done in a plugin system which is pretty cool. We can basically assign plugins for certain log levels (and more). This was the FileLogger. This all still does not explain why it failed with this particular error message. But a bit of digging in the Log/Plugin system revealed the issue.
register_shutdown_function(array(__CLASS__, 'Shutdown'));
Message are stored in a queue and written to file in a register shutdown function. Which was failing because the logfile location was not set correctly. This was causing an exception to be thrown inside the registered shutdown function. And therefore triggering this horrible error message. Be careful with register_shutdown functions.
Thijs Lensselink is a PHP developer, consultant and all out open source enthusiast.
He has over 12+ years of experience in building and maintaining web applications mostly
on linux/Unix/BSD platforms. Besides a full time job he does freelance work with his ...
read more
