<?xml version="1.0" encoding="utf-8" ?>
<?xml-stylesheet href="/blog/templates/default/atom.css" type="text/css" ?>

<feed 
   xmlns="http://www.w3.org/2005/Atom"
   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
   xmlns:dc="http://purl.org/dc/elements/1.1/"
   xmlns:admin="http://webns.net/mvcb/"
   xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
   xmlns:wfw="http://wellformedweb.org/CommentAPI/">
    
    <link href="http://www.schlueters.de/blog/feeds/atom10.xml" rel="self" title="Johannes Schlüter" type="application/atom+xml" />
    <link href="http://schlueters.de/blog/"                        rel="alternate"    title="Johannes Schlüter" type="text/html" />
    <link href="http://schlueters.de/blog/rss.php?version=2.0"     rel="alternate"    title="Johannes Schlüter" type="application/rss+xml" />
    <title type="html">Johannes Schlüter</title>
    <subtitle type="html">Always searching for Life, the Universe and Everything</subtitle>
    <icon>http://schlueters.de/blog/templates/default/img/s9y_banner_small.png</icon>
    <id>http://schlueters.de/blog/</id>
    <updated>2012-01-12T12:23:59Z</updated>
    <generator uri="http://www.s9y.org/" version="1.5.5">Serendipity 1.5.5 - http://www.s9y.org/</generator>
    <dc:language>en</dc:language>

    <entry>
        <link href="http://schlueters.de/blog/archives/165-Upcoming-talks.html" rel="alternate" title="Upcoming talks" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2012-01-12T10:54:40Z</published>
        <updated>2012-01-12T12:23:59Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=165</wfw:comment>
    
        <slash:comments>1</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=165</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/9-MySQL" label="MySQL" term="MySQL" />
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/165-guid.html</id>
        <title type="html">Upcoming talks</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>Over the last few weeks I had been quite silent, but that's about to change: Over the next few weeks I'll give a few presentations. Feel free to join any of those.</p> 
<ul> 
<li>January, 18th: <a href="http://www.mysql.com/news-and-events/web-seminars/display-680.html">Erstellung hochperformanter PHP-Anwendungen mit MySQL</a> (German)<br />MySQL Webinar, Online</li> 
<li>February, 9th: <a href="http://www.oracle.com/webapps/events/ns/EventsDetail.jsp?p_eventId=148478&amp;src=7314534&amp;src=7314534&amp;Act=240">MySQL Konnectoren</a> (German)<br />OTN Developer Day: MySQL, Frankfurt, Germany</li> 
<li>February 24th/25th: <a href="http://www.phpconference.co.uk/talk/php-under-hood">PHP under the hood</a> (English)<br />PHP UK Conference, London, UK</li> 
</ul> 
            </div>
        </content>
        <dc:subject>conferences</dc:subject>
<dc:subject>mysql</dc:subject>
<dc:subject>mysqlde</dc:subject>
<dc:subject>php</dc:subject>
<dc:subject>travel</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/164-High-Performance-PHP-Session-Storage-on-Scale.html" rel="alternate" title="High Performance PHP Session Storage on Scale" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2011-11-17T15:34:00Z</published>
        <updated>2011-11-19T17:59:03Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=164</wfw:comment>
    
        <slash:comments>8</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=164</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/9-MySQL" label="MySQL" term="MySQL" />
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/164-guid.html</id>
        <title type="html">High Performance PHP Session Storage on Scale</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>One of the great things about the HTTP protocol, besides <a href="http://tools.ietf.org/html/rfc2324">status code 418</a>, is that it's stateless. A web server therefore is not required to store any information on the user or allocate resources for a user after the individual request is done. By that a single web server can handle many many many different users easily, and well if it can't anymore one can add a new server, put a simple load balancer in front and scale out. Each of those web servers then handles its requests without the need for communication which leads to linear scaling (assuming network provides enough bandwidth etc.).</p> 
<p>Now the Web isn't used for serving static documents only anymore but we have all these fancy web apps. And those applications often have the need for a state. The most trivial information they need is the current user. HTTP is a great protocol and provides a way to do authentication which works well with its stateless nature - unfortunately this authentication is implemented badly in current clients. Ugly popups, no logout button, ... I don't have to tell more I think. For having nicer login systems people want web forms. Now the stateless nature of HTTP is a problem: The user may login and then browse around. On later requests it should still be known who that user is - with a custom HTML form based login alone this is not possible. A solution might be cookies. At least one might think so for a second. But setting a cookie &quot;this is an authorized user&quot; alone doesn't make sense as it could easily be faked. Better is to simply store a random identifier in a cookie and then keep a state information on the server. Then all session data is protected and only the user who knows this random identifier is authenticated. If this identifier is wisely chosen and hard to guess this works quite well. Luckily this is a mostly PHP- and MySQL-focused blog and as PHP is a system for building web applications this functionality is part of the core language: The <a href="http://php.net/session">PHP session module</a>.</p> 
<p>The session module, which was introduced in PHP 4, partly based on work on the famous phplib library, is quite a fascinating piece of code. It is open and extendable in so many directions but still so simple to use that everybody uses it, often newcomers learn about it on their first day in PHP land. Of course you can not only store the information whether the user is logged in but cache some user-specific data or keep the state on some transactions by the user, like multi-page forms or such.<br /></p> 
<p>In its default configuration session state will be stored on the web server's file system. Each session's data in its own file in serialized form. If the filesystem does some caching or one uses a ramdisk or something this can be quite efficient. But as we suddenly have a state on the web server we can't scale as easily as before anymore: If we add a new server and then route a user with an existing session to the new server all the session data won't be there. That is bad. This is often solved by a configuration of the load balancer to route all requests from the same user to the same web server. In some cases this works quite ok, but it is often seen that this might cause problems. Let's assume you want to take a machine down for maintenance. All sessions there will die. Or imagine there's a bunch of users who do complex and expensive tasks - then one of your servers will have a hard time, giving these users bad response times which feels like bad service, even though your other systems are mostly idle.</p> 
<p>A nice solution for this would be to store the sessions in a central repository which can be accessed from all web servers.</p> <br /><a href="http://schlueters.de/blog/archives/164-High-Performance-PHP-Session-Storage-on-Scale.html#extended">Continue reading "High Performance PHP Session Storage on Scale"</a>
            </div>
        </content>
        <dc:subject>memcache</dc:subject>
<dc:subject>memcached</dc:subject>
<dc:subject>mysql</dc:subject>
<dc:subject>mysql cluster</dc:subject>
<dc:subject>performance</dc:subject>
<dc:subject>php</dc:subject>
<dc:subject>php session</dc:subject>
<dc:subject>scalability</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/163-mysqli_result-iterations.html" rel="alternate" title="mysqli_result iterations" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2011-11-14T15:19:18Z</published>
        <updated>2011-11-17T11:19:58Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=163</wfw:comment>
    
        <slash:comments>2</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=163</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/9-MySQL" label="MySQL" term="MySQL" />
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/163-guid.html</id>
        <title type="html">mysqli_result iterations</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>For the last few months I had quite a few MySQL blog posts and didn't have anything from my &quot;<a href="http://schlueters.de/blog/plugin/tag/php%FFnext">new features in PHP [trunk|5.4]</a>&quot; series. This article is a bridge between those two. The <a href="http://svn.php.net/viewvc/php/php-src/branches/PHP_5_4/NEWS?revision=HEAD&amp;view=markup">PHP 5.4 NEWS</a> file has a small entry:</p> 
<p> </p> 
<blockquote>MySQLi: Added iterator support in MySQLi. mysqli_result implements Traversable. (Andrey, Johannes)<br /></blockquote> 
<p>From the outside it is a really small change and easy to miss. The mentioned class, mysqli_result, implements an interface which adds no new methods. What once can't see is that this relates to some internal C-level functions which can be called by the engine for doing a foreach iteration on objects of this class. So with PHP 5.4 you don't have to use an &quot;ugly&quot; while construct anymore to fetch rows from a mysqli result but can simply do a foreach:</p> 
<blockquote> 
<pre><a title="" target="" href="http://php.net/mysqli_report">mysqli_report</a>(MYSQLI_REPORT_STRICT);
try {
    $mysqli = new mysqli(/* ... */);
    foreach ($myslqi-&gt;<a title="" target="" href="http://php.net/mysqli.query">query</a>("SELECT a, b, c FROM t") as $row) {
        /* Process $row which is an associative array */
    }
} catch (mysqli_sql_exception $e) {
    /* an error happened ... */
}
</pre> 
</blockquote> 
<p>I'm configuring mysqli in a way to throw exceptions on error. This is useful in this case as mysqli::query() might return false in the case of an error. Passing false to a foreach will give a fatal error, so I'd need a temporary variable and a check in front of the foreach loop, with exceptions I simply do the error handling in the catch block.</p> 
<p>One thing to note is that mysqli is using buffered results (&quot;store result&quot;) by default. If you want to use unbuffered result sets (&quot;use result&quot;) you can easily do that by setting the flag accordingly:</p> 
<blockquote> 
<pre>foreach ($myslqi-&gt;query("SELECT a, b, c FROM t", MYSQLI_USE_RESULT) as $row) {
    /* ... */
}
</pre> 
</blockquote> 
<p>People who are advanced with iterators in PHP might ask &quot;Why did you implement Traversable only, not <a href="http://php.net/iterator">Iterator</a>?&quot; - the main reason is that we simply didn't want to. The mysqli_result class already has quite a few methods and we didn't want to make the interface confusing. If you need an Iterator class for some purpose you can simply wrap mysqli_result in an <a href="http://php.net/IteratorIterator">IteratoIterator</a>.<br /></p> 
            </div>
        </content>
        <dc:subject>mysql</dc:subject>
<dc:subject>mysqli</dc:subject>
<dc:subject>php</dc:subject>
<dc:subject>php 5.4</dc:subject>
<dc:subject>php oo</dc:subject>
<dc:subject>php.iterator</dc:subject>
<dc:subject>php.next</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/162-mysqlnd_qc-and-Symfony2.html" rel="alternate" title="mysqlnd_qc and Symfony2" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2011-10-07T20:52:00Z</published>
        <updated>2011-10-14T07:23:53Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=162</wfw:comment>
    
        <slash:comments>1</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=162</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/9-MySQL" label="MySQL" term="MySQL" />
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/162-guid.html</id>
        <title type="html">mysqlnd_qc and Symfony2</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>Previously I was writing about combining <a href="http://schlueters.de/blog/archives/161-Symfony-2-and-mysqlnd.html">Symfony2 and mysqlnd</a> to get more statistics on what is going on below the surface in the database communication when using a Symfony2 application via the Symfony2 profiler. Now that's not all that can be done and I gave some ideas for extending this. One idea was adding mysqlnd_qc support. mysqlnd_qc is the client side query cache plugin for mysqlnd. This provides a client-side cache for query results transparently without changing the application.</p> 
<p> 
<iframe width="670" height="550" src="/presentations/#p=20101030_barcelona_MySQL_Caching">&amp;amp;amp;lt;span id=&amp;amp;amp;quot;XinhaEditingPostion&amp;amp;amp;quot;&amp;amp;amp;gt;&amp;amp;amp;lt;/span&amp;amp;amp;gt;&amp;amp;amp;amp;amp;lt;span id=&amp;amp;amp;amp;amp;quot;XinhaEditingPostion&amp;amp;amp;amp;amp;quot;&amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;lt;/span&amp;amp;amp;amp;amp;gt;</iframe> 
</p> 
<p><a href="http://schlueters.de/blog/uploads/screenshots/symfony2_mysqlnd_qc.png" class="serendipity_image_link"><!-- s9ymdb:70 --><img style="width: 162px; height: 62px;" src="http://schlueters.de/blog/uploads/screenshots/symfony2_mysqlnd_qc.serendipityThumb.png" class="serendipity_image_right" /></a>A nice thing about this plugin, for this context here, is the function <a href="http://php.net/mysqlnd_qc_get_query_trace_log">mysqlnd_qc_get_query_trace_log()</a> which provides information about each query being executed. Not only the query string but also some timing (execution time, result storage time) and a stack trace so you can see where in the code a query was executed. I've added this functionality to the <a href="https://github.com/johannes/JSMysqlndBundle">JSMysqlndBundle</a> as you can see in the <a href="http://schlueters.de/blog/uploads/screenshots/symfony2_mysqlnd_qc.png" class="serendipity_image_link">screenshot</a>. I won't show a screenshot about what happens if you click the stacktrace link as this currently breaks the layout a bit, but maybe somebody wants to make this nicer? - Or maybe even feels motivated to make it even better using <a href="http://docs.php.net/manual/en/book.mysqlnd-uh.php">mysqlnd_uh</a> (which, as of today, has docs, thanks to Ulf) Feel free to contact me to talk about ideas! <img src="http://schlueters.de/blog/templates/default/img/emoticons/smile.png" alt=":-)" style="display: inline; vertical-align: bottom;" class="emoticon" /><br /></p> 
            </div>
        </content>
        <dc:subject>coding</dc:subject>
<dc:subject>mysqlnd</dc:subject>
<dc:subject>mysqlnd_qc</dc:subject>
<dc:subject>php</dc:subject>
<dc:subject>symfony2</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/161-Symfony-2-and-mysqlnd.html" rel="alternate" title="Symfony 2 and mysqlnd" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2011-10-02T12:23:00Z</published>
        <updated>2011-10-07T20:58:53Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=161</wfw:comment>
    
        <slash:comments>6</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=161</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/9-MySQL" label="MySQL" term="MySQL" />
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/161-guid.html</id>
        <title type="html">Symfony 2 and mysqlnd</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>In a <a href="http://schlueters.de/blog/archives/159-mysqlnd-plugins-and-json.html">previous blog posting</a> I was mentioning that I'm working on a small hobby PHP project. As I'm using this project to update myself to current frameworks I've decided to use Symfony2. Symfony provides a nice feature, which is the Symfony Profilier, an extensive logging and reporting system for Symfony2 developers to understand what's going on. A part of it is the Doctrine query logger which lists all database queries executed by Doctrine and their execution time.</p> 
<p><a href="http://schlueters.de/blog/uploads/screenshots/symfony_mysqlnd.png" class="serendipity_image_link"><!-- s9ymdb:69 --><img width="110" height="109" src="http://schlueters.de/blog/uploads/screenshots/symfony_mysqlnd.serendipityThumb.png" class="serendipity_image_left" /></a>This is nice but when we're using mysqlnd in our PHP build we have more information available. &quot;So why not use that information,&quot; I thought and built a new bundle for Symfony2 doing exactly that. The <a href="http://github.com/johannes/JSMysqlndBundle">JSMysqlndBundle</a> will take all the <a href="http://php.net/mysqlnd.stats">150 or so statistic values</a> collected, so they can be seen in the profiler (click screenshot for a larger view).</p> 
<p>As this is the initial value, a quick Sunday morning hack, it has not all features I can imagine. Things one could do include</p> 
<ul> 
<li>Provide information on caching decisions and behavior when <a href="http://php.net/mysqlnd_qc">mysqlnd_qc</a> is used</li> 
<li>Provide replication-related decisions when the new <a href="http://php.net/mysqlnd_ms">mysqlnd replication and load balancing plugin</a> is used</li> 
<li>Take David's <a href="http://blog.mayflower.de/archives/578-MySQLnd-Plugins-Writing-a-MySQL-Query-Logger-in-PHP.html">mysqlnd_uh-based query logging</a> ideas and provide more information on any executed query</li> 
<li>....</li> 
</ul>Any takers? - Code away!<br /> 
            </div>
        </content>
        <dc:subject>coding</dc:subject>
<dc:subject>mysqlnd</dc:subject>
<dc:subject>php</dc:subject>
<dc:subject>symfony2</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/150-MySQL-Query-Analyzer-and-PHP.html" rel="alternate" title="MySQL Query Analyzer and PHP" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2011-09-29T22:26:00Z</published>
        <updated>2011-09-29T23:35:57Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=150</wfw:comment>
    
        <slash:comments>2</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=150</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/9-MySQL" label="MySQL" term="MySQL" />
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/150-guid.html</id>
        <title type="html">MySQL Query Analyzer and PHP</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>Today we've released a few things for PHP and MySQL users: One is the first (and probably only) beta of the <a href="http://pecl.php.net/mysqlnd_ms">mysqlnd_ms plugin</a> for MySQL Replication and Load Balancing support and the first GA version of a PHP plugin for the Query Analyzer of the MySQL Enterprise Monitor.</p> 
<p>Ulf <a href="http://blog.ulf-wendel.de/">blogged a lot</a> about mysqlnd_ms, so I don't have to repeat him here. what I want to talk about is the other one. So what is that about?<br /></p> 
<p>When running a PHP-based application with MySQL it is often quite interesting to see what actually happens on the database sever. Besides monitoring of the system load etc. it is often interesting to see what queries are actually executed and which of them are expensive. A part of MySQL Enterprise Monitor is the <a href="http://mysql.com/products/enterprise/query.html">MySQL Query Analyzer</a> which helps answering these questions.</p> 
<p>Traditionally the MySQL Query Analyzer&#160; was based on MySQL Proxy which is configured to sit between the application and the MySQL server and collects all relevant data.<br /></p> 
<p>Now in the new <a href="http://blogs.oracle.com/mysqlenterprise/entry/mysql_enterprise_monitor_2_33">2.3.7 release</a> of the MySQL Enterprise Monitor we have enhanced this for PHP users: We now provide a plugin which can be loaded in PHP and which will provide data for the Query Analyzer directly from within PHP.</p> 
<p> <!-- s9ymdb:67 --><img width="1053" height="690" src="http://schlueters.de/blog/uploads/screenshots/mem_quan_php1.png" class="serendipity_image_center" style="max-width:100%" /></p> 
<p>By that we don't only reduce the latency for the data collection but we can provide more information about the current environment.</p> 
<p><!-- s9ymdb:68 --><img width="837" height="694" src="http://schlueters.de/blog/uploads/screenshots/mem_quan_php2.png" class="serendipity_image_center" /> </p> 
<p>In the query detail window you now don't only see general query statistics but also a stack trace from the application, so you can immediately identify the part of the application which should be improved. So above you can see a few screenshots I made from this server showing some insights of this blog where I was testing the plugin.</p> 
<p>If you want to learn more checkout the <a href="http://dev.mysql.com/doc/mysql-monitor/2.3/en/mem-qanal-using-cphp.html">documentation</a> and <a href="http://mysql.com/products/enterprise/monitor.html">product pages</a>. Hope you like it!<br /></p> 
            </div>
        </content>
        <dc:subject>cta</dc:subject>
<dc:subject>mysql</dc:subject>
<dc:subject>mysqlnd</dc:subject>
<dc:subject>mysqlnd plugins</dc:subject>
<dc:subject>query analyzer</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/160-Direct-MySQL-Stream-Access-Revised.html" rel="alternate" title="Direct MySQL Stream Access - Revised" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2011-09-26T10:56:00Z</published>
        <updated>2011-09-26T11:53:17Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=160</wfw:comment>
    
        <slash:comments>3</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=160</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/9-MySQL" label="MySQL" term="MySQL" />
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/160-guid.html</id>
        <title type="html">Direct MySQL Stream Access - Revised</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>Roughly three years ago I was writing about <a href="http://schlueters.de/blog/archives/86-Direct-MySQL-Stream-Access.html">Direct MySQL Stream Access</a> - a way to access the low-level stream PHP's mysqlnd library is using. Back then this had been a patch against PHP's mysqli extension. As such a feature is quite dangerous (you can easily mess with the connection state which confuses mysqlnd and/or the MySQL server) we didn't push it into the main PHP tree. Now three years later it's time to look at this again as we don't need to patch PHP anymore.</p> 
<p>Since the mentioned patch was written mysqlnd got a plugin interface about which <a href="http://schlueters.de/blog/plugin/tag/mysqlnd+plugins">I was talking before</a>. This plugin-interface, especially in the version of PHP 5.4, makes it trivial to implement this feature.</p> 
<blockquote> 
<pre>PHP_FUNCTION(mysqlnd_to_stream)
{
&#160;&#160;&#160; zval *conn_zv;
&#160;&#160;&#160; MYSQLND *conn;

&#160;&#160;&#160; if (zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "z", &amp;conn_zv) == FAILURE) {
&#160;&#160;&#160;&#160;&#160;&#160;&#160; return;
&#160;&#160;&#160; }

&#160;&#160;&#160; if (!(conn = zval_to_mysqlnd(conn_zv))) {
&#160;&#160;&#160;&#160;&#160;&#160;&#160; php_error_docref(NULL TSRMLS_CC, E_WARNING,
&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; "Passed variable is no mysqlnd based connection");
&#160;&#160;&#160;&#160;&#160;&#160;&#160; RETURN_FALSE;
&#160;&#160;&#160; }

&#160;&#160;&#160; php_stream_to_zval(conn-&gt;net-&gt;stream, return_value);
}
&#160;</pre> 
</blockquote> 
<p>If you take a function like the one shown above and add some general PHP infrastructure you are done. The key function here is the function <em>MYSQLND* zval_to_mysqlnd(zval *connection)</em> which takes a PHP variable as parameter and in case it is a MySQL connection (ext/mysql, mysqli or pdo_mysql) will return the corresponding MYSQLND pointer which gives access to the stream which then has to be packed into a PHP variable, again. The nice thing, compared to the old version is not only that it is a plugin which can be loaded into PHP as shared extension via php.ini but also that it works with all MySQL extensions, not only mysqli as the one before.</p> 
<p>You can <a href="http://schlueters.de/blog/uploads/code/mysqlnd_to_stream.tar.bz2">download the complete source</a>, but be warned: This is experimental stuff and not supported in any way, but I hope good enough to get a feeling what's possible with mysqlnd.<br /></p> 
            </div>
        </content>
        <dc:subject>coding</dc:subject>
<dc:subject>mysqlnd</dc:subject>
<dc:subject>mysqlnd plugins</dc:subject>
<dc:subject>php</dc:subject>
<dc:subject>php extensions</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/159-mysqlnd-plugins-and-json.html" rel="alternate" title="mysqlnd plugins and json" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2011-09-14T11:27:00Z</published>
        <updated>2011-09-16T07:56:15Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=159</wfw:comment>
    
        <slash:comments>3</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=159</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/9-MySQL" label="MySQL" term="MySQL" />
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/159-guid.html</id>
        <title type="html">mysqlnd plugins and json</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>Some time ago I was already writing about the <a href="http://schlueters.de/blog/archives/146-mysqlnd-plugins-for-PHP-in-practice.html">power included with mysqlnd plugins</a> and how they can they can be used transparently to help you with your requirements without changing your code. But well, as mysqlnd plugins in fact are regular PHP extensions they can export functions to the PHP userland and providing complete new functionality.</p> 
<p>In my spare time I'm currently writing a shiny Web 2.0 application where I'm heavily using AJAX-like things, so what I do quite often in this application is, basically this: Check some pre-conditions (permissions etc.) then select some data from the database, do a fetch_all to get the complete result set as an array and run it through json_encode; or to have it in code:</p> 
<blockquote> 
<pre>&lt;?php
$m = new MySQLi(/*...*/);
check_whether_the_user_is_checked_in_and_allowed_to_see_this();
$result = $m-&gt;query("SELECT a,b,c,d FROM t WHERE e=23");
echo json_encode($result-&gt;fetch_all());
?&gt;
</pre> 
</blockquote> 
<p>Of course that example is simplified as I'm using the Symfony 2 framework for this project. When writing a similar function for the 5th time I wondered whether I really need to create the temporary array and all these temporary elements in it.</p> 
<p>So I wrote a mysqlnd plugin.</p> 
<p>The mysqlnd_query_to_json plugin (hey what a name!) provides a single function, mysqlnd_query_to_json(), which takes two parameters, a connection identifier and an SQL query, and returns a JSON string containing the result set. The connection identifier can be a mysql resource, a mysqli object or even a PDO object. The resulting JSON string will be created directly from the network buffer without the need of temporary complex structures. Using the above example would create code like this:</p> 
<blockquote> 
<pre>&lt;?php
$m = new MySQLi(/*...*/);
check_whether_the_user_is_checked_in_and_allowed_to_see_this();
echo mysqlnd_query_to_json($m, "SELECT a,b,c,d FROM t WHERE e=23");
?&gt;
</pre> 
</blockquote> 
<p>The plugin, which you can <a href="http://schlueters.de/blog/uploads/code/mysqlnd_query_to_json.tar.bz2">find here</a>, requires PHP 5.4 and has a few limitations as it knows nothing about MySQL bitfields or escaping of unicode characters for creating fully valid JSON data and Andrey called it, for good reasons, a hack. Neither did I benchmark it, yet as I merely share it to show what's possible and maybe start some discussion on what is actually needed.</p> 
<p>If you want to learn more on these topics I also suggest to check the <a href="http://www.mysql.com/news-and-events/web-seminars/">MySQL Webinar page</a> frequently as Ulf is going to hold a Webinar on myslqnd plugins in October!<br /></p> 
            </div>
        </content>
        <dc:subject>coding</dc:subject>
<dc:subject>mysql</dc:subject>
<dc:subject>mysqlnd</dc:subject>
<dc:subject>mysqlnd plugins</dc:subject>
<dc:subject>php</dc:subject>
<dc:subject>php extensions</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/156-Improvements-for-PHP-application-portability-in-PHP.next.html" rel="alternate" title="Improvements for PHP application portability in PHP.next" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2011-07-25T20:44:00Z</published>
        <updated>2011-07-30T17:29:29Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=156</wfw:comment>
    
        <slash:comments>7</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=156</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/156-guid.html</id>
        <title type="html">Improvements for PHP application portability in PHP.next</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>I was writing about <a href="http://schlueters.de/blog/plugin/tag/php%FFnext">PHP.next before</a>, many things improved there meanwhile. Most notably we have a committed version number: The next PHP release will be called PHP 5.4. The topic I want to talk about today is &quot;Improved application portability&quot; which covers multiple small changes which aim at making it simpler for developers to write applications working on any PHP setup.</p> 
<p><strong>Separating <em>&lt;?=</em> from <em>short_open_tags</em></strong></p> 
<p>PHP knows quite a few ways to separate PHP code from surrounding text (usually HTML), most applications use <em>&lt;?php</em> as that works on every system. There is a short form of this, <em>&lt;?</em>, which can be disabled using php.ini's short_open_tags setting. Being able to disable this is important when embedding PHP code into XML documents containing XML processing instructions. Now we also have <em>&lt;?=</em> which, basically, is a shortcut for <em>&lt;?php echo</em>. This tag is useful when using PHP as templating language as it prevents cluttered code. The issue in current version of PHP is that this is bound to short_open_tags, so portable applications can't rely on it. But PHP 5.4 will bring the solution: <em>&lt;?=</em> will always be there, independently from short_open_tags. Yay!</p> 
<p><strong>No more magic_quotes</strong></p> 
<p>In the old times it was easy to write code using PHP.</p> 
<blockquote> 
<p>&lt;?php<br />$q = mysql_query(&quot;SELECT * FROM t WHERE name = '$name' &quot;);<br />?&gt;</p> 
</blockquote> 
<p>And you had, thanks to register_globals, some data to work on and this was mostly secure as PHP automatically escaped request data. But well this escaping worked only in a few cases acceptable good. Besides not knowing anything about other encodings or DBMS-specific escape sequences it also failed for non-string values as in</p> 
<blockquote> 
<p>&lt;?php<br />
$q = mysql_query(&quot;SELECT * FROM t WHERE id = $id &quot;);<br />
?&gt;</p> 
</blockquote> 
<p>Where the external value wasn't escaped. So portable applications, which aim at being secure nowadays have to check whether magic_quote_gpc is enabled, then remove the &quot;bad&quot; quotes and then finally escape again using the appropriate way. That's quite&#160; an annoyance and doing this the wrong way can cause bad bugs (like forcing such a replacement logic in an endless recursion by providing arrays) So nobody really likes magic_quotes. So with PHP 5.4 they are gone. No more need to worry about them. Use the proper escaping and you're done. Wonderful. Only issue: Legacy applications might rely on magic_quotes so when upgrading PHP make sure the application does the required escaping itself so almost-secure applications won't become insecure.</p> 
<p><strong>Dropped explicit --enable-zend-multibyte compile-time option</strong></p> 
<p>Especially in Asia people use multi-byte encodings which aren't ASCII-compatible so mixing them with PHP code might be hard. In current versions of PHP there is a compile-time option to enable a special multibyte mode for the engine which will handle this in the engine so PHP code can be provided using these encodings. By this portable applications had a hard time due to this conversion (not) being done. Thanks to the work by Dmitry and Moriyoshi this mode is now always enabled whithout penalty for people not depending on it and the extended functionality from mbstring can be provided as a shared module. By this distributors can provide a single build which will work for everybody.</p> 
<p><strong>Closing remarks</strong> <br /></p> 
<p>As always in this series: Be aware that things discussed here might change. Please try out the current snapshot of PHP 5.4 and test it with your applications. No we can still fix backwards compatibility breaks. fixing them after a release will possibly break it for people depending on the new behavior. Happy coding!<br /></p> 
            </div>
        </content>
        <dc:subject>coding</dc:subject>
<dc:subject>php</dc:subject>
<dc:subject>php 5.4</dc:subject>
<dc:subject>php.next</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/157-OSCON-2011.html" rel="alternate" title="OSCON 2011" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2011-06-30T22:41:00Z</published>
        <updated>2011-07-04T07:15:51Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=157</wfw:comment>
    
        <slash:comments>4</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=157</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/157-guid.html</id>
        <title type="html">OSCON 2011</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>This year I'll attend OSCON for the first time. I'll give two talks:</p> 
<ul> 
<li><a href="http://www.oscon.com/oscon2011/public/schedule/detail/18799">PHP and MySQL - Recent Developments</a><br />PHP’s MySQL support recently received many changes under the hood. PHP 5.3 introduced mysqlnd – the MySQL native driver which is a replacement for libmysql deeply bound into PHP. In this presentation you will learn what the PHP and MySQL development teams were up to. After starting with an introduction of the PHP-stack, demystifying things like mysqli, mysqlnd or PDO, this presentation will show you how to build mysqlnd plugins as PHP C extension and hooking into mysqlnd from PHP  userland. It will also discuss existing plugins like a client side query cache or a module for doing read-write-splitting, both working transparently, without changes to your application.</li> 
<li><a href="http://www.oscon.com/oscon2011/public/schedule/detail/18904">PHP Under the hood</a><br />
The beauty of PHP is that everybody can read the code and see the inner workings of software. But understanding concepts from reading code isn’t often helpful. Especially if you are no pro in that language. This presentation will take apart many parts of the PHP runtime, describe the concepts behind so attendees understand the inner workings without actually reading C code. Concepts covered include HashTables, the foundation for PHP  arrays and many other internal data structures, the reference counting mechanism, which is important for writing efficient code as well as the overall executor.</li> 
</ul> 
<p>In case you can't make it to these talks but want to talk to me you'll probably find me at the Oracle booth where I'll also try to give some short talks on some topics to be defined (any wishes?)</p> 
<p>&#160;In case you're not interested in me and my talks but MySQL there are a few sessions by other MySQL Engineers:</p> 
<ul> 
<li><a href="http://www.oscon.com/oscon2011/public/schedule/detail/19057">MySQL Replication Update</a></li> 
<li><a href="http://www.oscon.com/oscon2011/public/schedule/detail/19223">InnoDB: Performance and Scalability Features</a></li> 
<li><a href="http://www.oscon.com/oscon2011/public/schedule/detail/18788">Python Utilities for Managing MySQL Databases</a></li> 
<li><a href="http://www.oscon.com/oscon2011/public/schedule/detail/18785">The MySQL Time Machine</a></li> 
</ul>In case you want to attend you can use the code <em>os11fos</em> which should give you a 20% discount. <br /> 
            </div>
        </content>
        <dc:subject>conferences</dc:subject>
<dc:subject>mysql</dc:subject>
<dc:subject>oscon</dc:subject>
<dc:subject>php</dc:subject>
<dc:subject>travel</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/158-PHP-5.4-Alpha-1.html" rel="alternate" title="PHP 5.4 Alpha 1" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2011-07-02T00:23:00Z</published>
        <updated>2011-07-04T07:07:26Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=158</wfw:comment>
    
        <slash:comments>1</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=158</wfw:commentRss>
    
    
        <id>http://schlueters.de/blog/archives/158-guid.html</id>
        <title type="html">PHP 5.4 Alpha 1</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>Recently <a href="http://qa.php.net">PHP 5.4.0 Alpha 1</a> was released and the PHP development team is asking every PHP user to test it. In this blog I have some articles about upcoming features in that version. Now is a good time to test 5.4 in combination with your applications spot mistakes (complain now if we break compatibility, now we could fix it ...) and a good time to prepare your knowledge.</p> 
<p>These are the articles I published here:</p> 
<ul> 
<li><a href="http://schlueters.de/blog/archives/129-Mind-the-encodings!.html">Mind the encoding</a></li> 
<li><a href="http://schlueters.de/blog/archives/133-Now-in-trunk-Improved-interactive-shell.html">Improved interactive shell</a></li> 
<li><a href="http://schlueters.de/blog/archives/135-Jason,-let-me-help-you!.html">Jason, let me help you</a></li> 
<li><a href="http://schlueters.de/blog/archives/138-Features-in-PHP-trunk-Array-dereferencing.html">Array dereferencing</a></li> 
<li><a href="http://schlueters.de/blog/archives/147-Changes-in-PHP-trunk-No-more-extension-for-sqlite-version-2.html">No more extension for SQLite 2</a></li> 
<li><a href="http://schlueters.de/blog/archives/151-Upload-Progress-in-PHP-trunk.html">Upload progress</a></li> 
</ul> 
<p>I also have articles on a feature which does not make that release:</p> 
<ul> 
<li><a href="http://schlueters.de/blog/archives/139-Scalar-type-hints-in-PHP-trunk.html">Scalar type hints in PHP trunk</a></li> 
<li><a href="http://schlueters.de/blog/archives/148-More-on-scalar-type-hints-in-PHP-trunk.html">More on scalar type hints in PHP</a></li> 
</ul>I plan to continue that series, focusing on things which might be overseen easily. It's a bit time till 5.4 will be released as GA but the more you test it and give feedback the better it will be! <img src="http://schlueters.de/blog/templates/default/img/emoticons/smile.png" alt=":-)" style="display: inline; vertical-align: bottom;" class="emoticon" /><br /> 
            </div>
        </content>
        <dc:subject>coding</dc:subject>
<dc:subject>php</dc:subject>
<dc:subject>php 5.4</dc:subject>
<dc:subject>php qa</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/155-Escaping-from-the-statement-mess.html" rel="alternate" title="Escaping from the statement mess" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2011-05-18T13:00:00Z</published>
        <updated>2011-05-22T16:59:55Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=155</wfw:comment>
    
        <slash:comments>22</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=155</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/155-guid.html</id>
        <title type="html">Escaping from the statement mess</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>One of the issues Web Developers face is making their application robust to prevent SQL injection attacks. Different approaches exist which help. Sometimes people use large abstraction layers (which, sometimes, don't make anything safe ...) and sometimes people use prepared statements as a way to secure queries. Now prepared statements were a nice invention some 30 years ago abut they weren't meant for making things secure and so they do have some shortcomings: One issue is that preparing and executing a query adds a round-trip to the server where it then requires resources. In a <em>classic</em> application this is no issue. The users starts the application up early in the morning and processes data multiple times so the prepared statement handle is re-used quite some time. The system benefits from early optimisations.&#160; In a typical PHP Web application this isn't the case. A request and therefore a database connection with its associated statement handles lives way less than a second before being thrown away. The PDO MySQL driver, by default, tries to improve that by emulating the prepared statement on the client side. This emulation faces issues as it is lacking the knowledge of what's valid SQL which can lead to strange behaviour (The simple example is $pdo-&gt;prepare(&quot;SELECT * FROM t LIMIT ?&quot;)-&gt;execute(array($_GET['count'])); which will emit an SQL syntax error) and inherits limitations from prepared statements. A second issue with prepared statements is that queries are being built dynamically. A common case which is hard to do with prepared statements is the IN() clause with a dynamic amount of values. With prepared statements you first have to build the list of place holders (the exact amount of place holders (<em>?</em>) separated by a comma, without trailing comma)&#160; and then bind the values and mind the offsets when having other values - this typically becomes ugly code.</p> 
<p>So why not take a step back. - Let's not try to emulate prepared statements but try to make it simpler to construct queries while escaping data?</p> 
<p>An API for doing this might follow the sprintf() semantics and look like this;</p> 
<blockquote> 
<pre>$sql = mysqli_format_query($mysqli, "SELECT * FROM t WHERE f1 = %s AND f2 = %i", "foobar", 23);
</pre> 
</blockquote> 
<p>which would return a string</p> 
<blockquote> 
<pre>SELECT * FROM t WHERE fi = 'foobar' AND f2 = 23
</pre> 
</blockquote> 
<p>which can safely be send to the database. As said the IN clause should work. as we're in PHP we might simply extend it to do this:</p> 
<blockquote> 
<pre>$sql = mysqli_format_query($mysqli, "SELECT * FROM t WHERE f1 IN (%s)", array("foobar", 23));

SELECT * FROM t WHERE f1 IN ('foobar', '23')
</pre> 
</blockquote> 
<p>Well doesn't look fancy? - But there's more: By not pretending to emulate prepared statements we can easily work with more dynamic queries. Something along the lines of</p> 
<blockquote> 
<pre>$sql = mysqli_format_query($mysqli, "SELECT * FROM t WHERE uid = %i", $_SESSION['uid']);
if (isset($option['option1']) {
    $sql .= mysqli_format_query($mysqli, "AND option1 = %s", $option['option1']);
}
if (isset($option['option2']) {
    $sql .= mysqli_format_query($mysqli, "OR option2 = %s", $option['option2']);
}
</pre> 
</blockquote> 
<p>Doing such a thing using prepared statements&#160; or in some <em>classic</em> way becomes way harder to maintain. For playing with this approach I quickly cooked up a simple implementation of that logic which should work well with PHP 5.3 and mysqli:<br /></p> <br /><a href="http://schlueters.de/blog/archives/155-Escaping-from-the-statement-mess.html#extended">Continue reading "Escaping from the statement mess"</a>
            </div>
        </content>
        <dc:subject>coding</dc:subject>
<dc:subject>mysql</dc:subject>
<dc:subject>php</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/154-Firefox-Add-ons.html" rel="alternate" title="Firefox Add-ons" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2011-05-14T10:26:00Z</published>
        <updated>2011-05-14T13:31:19Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=154</wfw:comment>
    
        <slash:comments>2</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=154</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/2-Software" label="Software" term="Software" />
    
        <id>http://schlueters.de/blog/archives/154-guid.html</id>
        <title type="html">Firefox Add-ons</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>
Modular software can be pain - you end up installing tons of add-opns. I recently configured a new desktop box and had to reconfigure Firefox. Here's the lsit of extensions I installed, I'd be interested in comments about better or missing add-ins</p> 
<ul> 
<li><strong>&#160;Adblock Plus</strong><br />This is a tricky one. I get content for free for looking at some ads. Might be a fair deal, but often it's too much. On some sites I allow ads, on others not.</li> 
<li><strong>BetterPrivacy</strong><br />Prevent Flash Cookies from tracking me</li> 
<li><strong>Brief</strong><br />An RSS/Atom/... Feed reader</li> 
<li><strong>Certificate Patrol</strong><br />Notifies me when sites change their SSL certifactes, might prevent fraud.</li> 
<li><strong>Change Rerer Button</strong><br />I don't like being tracked. At least not always <img src="http://schlueters.de/blog/templates/default/img/emoticons/smile.png" alt=":-)" style="display: inline; vertical-align: bottom;" class="emoticon" /></li> 
<li><strong>CookieSafe</strong><br />Firefox's cookie handling is damn limited. With this add-in I can edit/delete single cookies way simpler and easily change the per site settings</li> 
<li><strong>Firebug</strong><br />Good for fixing broken sites</li> 
<li><strong>FoxyProxy Standard</strong><br />When switching between company VPN and my local net I need different proxy configuration. With this extension I don'T have to go through the settings</li> 
<li><strong>Greasemonkey</strong><br /><em>Improve</em> websites according to my interest</li> 
<li><strong>JSON View</strong><br />Renders json more nicely. Good when working with JSON-based protocols</li> 
<li><strong>NoScript</strong><br />per-site configuration of javascript and stuff. Some websites have nice JS free versions.</li> 
<li><strong>Open in Browser</strong><br />Adds a new option to the download dialog to open in Browser as web page or plain text. Useful for files a server sends with a &quot;wrong&quot; header.</li> 
<li><strong>Tab Kit</strong><br />The wider the screen the more I want to have the tabs on the side. This add-in shows tabs in a sidebar with a tree structure and different colors and such</li> 
<li><strong>Tamper Data</strong><br />Edit HTTP Request before the browser sends them, initially I've got this for security research, nowadays I use it to work-around issues in bad WEb-UIs</li> 
<li><strong>User Agent Switcher</strong><br />Maybe switching the User agent every now and then makes tracking a bit harder, certainly it can be nice to get a different view. Many sites have a mobile version which is focussed on the content way mor than the regular site.</li> 
<li><strong>Web Developer</strong><br />Nice tool for working with HTML stuff<br /></li> 
</ul> 
            </div>
        </content>
        <dc:subject>add-ins</dc:subject>
<dc:subject>desktop</dc:subject>
<dc:subject>firefox</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/152-Not-only-SQL-memcache-and-MySQL-5.6.html" rel="alternate" title="Not only SQL - memcache and MySQL 5.6" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2011-04-14T12:04:28Z</published>
        <updated>2011-04-15T14:49:25Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=152</wfw:comment>
    
        <slash:comments>1</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=152</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/152-guid.html</id>
        <title type="html">Not only SQL - memcache and MySQL 5.6</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>This week there are two big events for the MySQL community: The <a href="http://mysqlconf.com">O'Reilly MySQL Conference</a> and <a href="http://collaborate11.ioug.org/">Oracle Collaborate</a> run by the IOUG. At these events our Engineering VP, Tomas Ulin, announced the latest milestone releases for our main products. MySQL 5.6 and MySQL Cluster 7.1 as well as our new Windows Installer. There's lots of cool stuff in there but one feature really excited me: MySQL 5.6 contains a memcache interface for accessing InnoDB tables. This means you can access data stored in MySQL not only using SQL statements but also by using a well established and known noSQL protocol.</p> 
<p>This works by having the <a href="http://www.memcached.org/">memcache daemon</a> running as plugin as part of the MySQL server. This daemon can then be configured in three ways: Either<br /></p> 
<ul> 
<li>to do what memcached always did - use an in memory hash table to store its data - or</li> 
<li>to access an InnoDB table to store and read data from or</li> 
<li>to use its own hash table in memory and fall back to InnoDB if data is not found directly in memcache.</li> 
</ul> 
<p>This combines the power of MySQL and InnoDB's persistent storage with the lightweight protocol memcache uses, which has faster connecting times (no authorization handshake etc.) and faster data access (no SQL parsing, optimization etc.) while you're still able to query the data using SQL when you're doing more complex operations.</p> 
<p>Of course I had to give it a run with PHP.</p> 
<p>First step for using this is fetching the MySQL preview release and configuring it accordingly. My colleague Jimmy Yang from the InnoDB team has <a href="http://blogs.innodb.com/wp/2011/04/get-started-with-innodb-memcached-daemon-plugin/">a nice blog posting</a> showing these first steps. After that we have to configure PHP where we have two choices: We can use the a bit older <a href="http://php.net/memcache">memcache</a> module or the newer <a href="http://php.net/memcached">memcached</a> module. I've chosen the first one as that was already configured on my system. On most systems the installation should be as easy as querying your package manager or using PECL:</p> 
<blockquote> 
<pre># pecl install memcache
or
# pecl install memcached</pre> 
</blockquote> 
<p>And then adding the corresponding entry (<em>extension=memcache[d].so</em>) to your php.ini file.<br /></p> 
<p>So let's do a first test from command line:</p> 
<blockquote> 
<pre>$ php -r '$m = memcache_connect("localhost", 11211); ' \
&#160;&#160;&#160;&#160; &#160; &#160; '$m-&gt;add("key", "value"); var_dump($m-&gt;get("key"));'
string(5) "value"
</pre> 
</blockquote> 
<p>So we store a value in memcache and then load it again to see if it was stored properly. Now we verify the results directly in MySQL:</p> 
<blockquote> 
<pre>mysql&gt; SELECT * FROM demo_test WHERE c1 = 'key';
Empty set (0.00 sec)
</pre> 
</blockquote> 
<p>Uh, what's wrong? - O simple: We didn't read Jimmy's article properly:</p> 
<blockquote> 
<p>If you would like to take a look at what’s in the “demo_test” table, 
please remember we had batched the commits (32 ops by default) by 
default. So you will need to do “read uncommitted” select to find the 
just inserted rows</p> 
</blockquote> 
<p>So we can apply that knowledge and query again:</p> 
<blockquote> 
<pre>mysql&gt; set session TRANSACTION ISOLATION LEVEL read uncommitted;
Query OK, 0 rows affected (0.00 sec)

mysql&gt; SELECT * FROM demo_test WHERE c1 = 'key';
+------+------+------+------+-------+------+------+------+------+------+------+
| cx   | cy   | c1   | cz   | c2    | ca   | CB   | c3   | cu   | c4   | C5   |
+------+------+------+------+-------+------+------+------+------+------+------+
| NULL | NULL | key  | NULL | value | NULL | NULL |    0 | NULL |    1 | NULL |
+------+------+------+------+-------+------+------+------+------+------+------+
1 row in set (0.00 sec)
</pre> 
</blockquote> 
<p>And yay! - We see our <em>value</em> in between the other columns for meta-data and other things.</p> 
<p>Both PHP modules provide a session handler so you can store your session data easily in <del>memcache</del>InnoDB. For configuring this we first need to add two entries to our php.ini file:</p> 
<blockquote> 
<pre>; when using the "memcache" extension:
session.save_handler=memcache
; when using the "memcached" extension:
; session.save_handler=memcached

session.save_path="tcp://localhost:11211"
</pre> 
</blockquote> 
<p>After restarting the web server, so it reads the new configuration we can test it with a simple script:</p> 
<blockquote> 
<pre>&lt;?php
session_start();
echo "&lt;pre&gt;Session ID: ".session_id()."\n";
var_dump($_SESSION);
$_SESSION['foo'] = 'bar';
?&gt;
</pre> 
</blockquote> 
<p>When first requesting this we will receive an output like</p> 
<blockquote> 
<pre>m1h4iqmp6hc7e4l85qlld0gtd
array(0) {
}
</pre> 
</blockquote> 
<p>Then we reload the page and see:</p> 
<blockquote> 
<pre>m1h4iqmp6hc7e4l85qlld0gtd1
array(1) {
  ["foo"]=&gt;
  string(3) "bar"
}</pre> 
</blockquote> 
<p>After that we can, again, look directly into MySQL:</p> 
<blockquote> 
<pre>mysql&gt; select * from demo_test where c1 = 'm1h4iqmp6hc7e4l85qlld0gtd1';
+------+------+----------------------------+------+----------------+------+------+------+------+------+------+
| cx   | cy   | c1                         | cz   | c2             | ca   | CB   | c3   | cu   | c4   | C5   |
+------+------+----------------------------+------+----------------+------+------+------+------+------+------+
| NULL | NULL | m1h4iqmp6hc7e4l85qlld0gtd1 | NULL | foo|s:3:"bar"; | NULL | NULL |    0 | NULL |    4 | NULL |
+------+------+----------------------------+------+----------------+------+------+------+------+------+------+
1 row in set (0.00 sec)

</pre> 
</blockquote> 
<p>I hope this helps you to get started. If you'd like to learn more about MySQL 5.6, MySQL Cluster 7.1 (which btw, also can be access using memcache!) our new installer or such you can watch a <a href="http://www.youtube.com/watch?v=L1V5T5rknq0&amp;p=4E26834501437A46">recording of Tomas' keynote</a> or visit <a href="http://dev.mysql.com">dev.mysql.com</a>.<br /></p> 
            </div>
        </content>
        <dc:subject>memcache</dc:subject>
<dc:subject>memcached</dc:subject>
<dc:subject>mysql</dc:subject>
<dc:subject>mysql56</dc:subject>
<dc:subject>oracle</dc:subject>
<dc:subject>php</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/151-Upload-Progress-in-PHP-trunk.html" rel="alternate" title="Upload Progress in PHP trunk" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2010-12-03T18:31:40Z</published>
        <updated>2010-12-07T04:24:43Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=151</wfw:comment>
    
        <slash:comments>10</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=151</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/151-guid.html</id>
        <title type="html">Upload Progress in PHP trunk</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>File uploads via HTTP are an annoyance. Web Browser know quite a lot but still the give little feedback to the users. Some a bit more, most close to no feedback. Now over the years this led to man unhappy users. Over the last year, with all these AJAX things, solutions emerged so that one can periodically poll the web server on a second connection for the status. For implementing this we have one architectural problem: PHP implements, for very good reasons, a shared nothing architecture. So one request from connection has no insight into another request/connection - but this is needed for the upload progress. Different people thought about this and implemented solutions. One of the first things was <a href="http://php.net/manual/en/apc.configuration.php#ini.apc.rfc1867">implemented in APC</a>, another one by a special <a href="http://pecl.php.net/package/uploadprogress">uploadprogress extension</a>. They are nice and found quite some adoption but they have two problems. For one they are not fully native to PHP, so they have to be installed additionally, and the use a local storage to transport the status. APC uses the system's shared memory. upload_progress the filesystem (yes, file systems could be shared, I do know that). not very satisfying for having PHP as <em>the</em> language for solving <em>the web problem</em>. </p> 
<p>The obvious solution, of course, would be to use PHP's <a href="http://php.net/session">session handling system</a> for this. The PHP session system is an integral part of PHP and can be configured to use different storage handlers, like the local file system or memcache, which can be useful to share session data in a load-balanced cluster. Now there were some technical issues why this wasn't done at first ... but then Arnaud Le Blanc sat down and created a proper implementation of an upload progress storage handler which has been commit to PHP trunk.</p> 
<p>Long story short: In the next version of PHP (5.4?) you will, mot likely, have an Upload Progress mechanism built-in.</p> 
<p>Arnaud wrote <a href="http://wiki.php.net/rfc/session_upload_progress">a nice RFC</a> explaining this functionality. So we configure our PHP to enable this feature, by making sure we have the default values:</p> 
<blockquote> 
<pre>session.upload_progress.enabled = 1 
session.upload_progress.prefix = upload_progress_
session.upload_progress.name = PHP_SESSION_UPLOAD_PROGRESS
</pre> 
</blockquote> 
<p>And we are set to go. Then we obviously need an HTML file upload form:</p> 
<blockquote> 
<pre><span class="sy0">&lt;</span>form action<span class="sy0">=</span><span class="st0">"upload.php"</span> method<span class="sy0">=</span><span class="st0">"POST"</span> enctype<span class="sy0">=</span><span class="st0">"multipart/form-data"</span><span class="sy0">&gt;</span>
 <span class="sy0">&lt;</span>input type<span class="sy0">=</span><span class="st0">"hidden"</span> name<span class="sy0">=</span><span class="st0">"&lt;?php echo ini_get("</span>session<span class="sy0">.</span>upload_progress<span class="sy0">.</span>name<span class="st0">"); ?&gt;"</span> value<span class="sy0">=</span><span class="st0">"johannesupload"</span> <span class="sy0">/&gt;</span>
 <span class="sy0">&lt;</span>input type<span class="sy0">=</span><span class="st0">"file"</span> name<span class="sy0">=</span><span class="st0">"file1"</span> <span class="sy0">/&gt;</span>
 <span class="sy0">&lt;</span>input type<span class="sy0">=</span><span class="st0">"file"</span> name<span class="sy0">=</span><span class="st0">"file2"</span> <span class="sy0">/&gt;</span>
 <span class="sy0">&lt;</span>input type<span class="sy0">=</span><span class="st0">"submit"</span> <span class="sy0">/&gt;</span>
<span class="sy0">&lt;/</span>form<span class="sy0">&gt;</span>
</pre> 
</blockquote> 
<p>And we can upload a file. If the file is big enough (and the connection slow enough) we can then periodically poll the server and read and read data about the progress from the <em>$_SESSION[&quot;upload_progress_johannesupload&quot;]</em> variable. The full contents is mentioned in the RFC, so I won't quote it here.</p> 
<p>So far so good. But how to poll? Well, take <a href="http://progphp.com/upload.phps">the sample by Rasmus</a> about the above mentioned APC-based solution and adopt it. I let this to the experienced reader as exercise <img src="http://schlueters.de/blog/templates/default/img/emoticons/smile.png" alt=":-)" style="display: inline; vertical-align: bottom;" class="emoticon" /></p> 
<p>Disclaimer: This article is describing features in not released software. Things may change without notice till it is release. Feel free to read other blog postings from <a href="http://schlueters.de/blog/plugin/tag/php%FFnext">my series of new features</a>!</p> 
<p>P.S. I still think the browser should give better feedback by itself. As should it offer better integration of HTTP-Auth, like a simple logout button ...<br /></p> 
            </div>
        </content>
        <dc:subject>apc</dc:subject>
<dc:subject>file upload</dc:subject>
<dc:subject>php</dc:subject>
<dc:subject>php session</dc:subject>
<dc:subject>php.next</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/148-More-on-scalar-type-hints-in-PHP-trunk.html" rel="alternate" title="More on scalar type hints in PHP trunk" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2010-11-23T06:05:00Z</published>
        <updated>2010-11-29T22:46:24Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=148</wfw:comment>
    
        <slash:comments>19</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=148</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/148-guid.html</id>
        <title type="html">More on scalar type hints in PHP trunk</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>Some <a href="http://schlueters.de/blog/archives/139-Scalar-type-hints-in-PHP-trunk.html">time ago I wrote an article</a> about the implementation of type hints for non-object types for PHP. Meanwhile many things happened and that implementation was replaced by a different one. Readers of my previous post might know that I have doubts about type hints in PHP. People who met me in person and asked me about it know for sure <img src="http://schlueters.de/blog/templates/default/img/emoticons/smile.png" alt=":-)" style="display: inline; vertical-align: bottom;" class="emoticon" /></p> 
<p>So what's the status now? - Well type hints, for non-object types, exist and they don't. There is a valid syntax which looks like this:</p> 
<blockquote> 
<pre>function foo(int $i) {
&#160;&#160;&#160; echo $i;
}
</pre> 
</blockquote> 
<p>What's the consequence of this code? - Well the type hint is simply ignored. This means that<br /></p> 
<blockquote> 
<pre>&#160;foo("Hello world");
</pre> 
</blockquote> 
<p>Will run without any error and print <em>Hello world</em>. So there is an syntax looking like another part of the language which throws errors but behaves completely different.</p> 
<blockquote> 
<pre>function foo(bar $b) {  }

foo("bar");'

Catchable fatal error: Argument 1 passed to foo() must be an instance of bar, string given [...]
</pre> 
</blockquote> 
<p>The <em>int</em> hint is just one of them, there are a few more:</p> 
<ul> 
<li>bool, boolean</li> 
<li>string, binary</li> 
<li>scalar</li> 
<li>numeric</li> 
<li>int, integer, long</li> 
<li>real, double, float</li> 
<li>resource</li> 
<li>object</li> 
</ul>So why is there a syntax added which is ignored? Well there are two things one can do: The hint can be read via the <a href="http://php.net/tokenizer">tokenizer</a> to do some magic or one might create a PHP extension which hooks into the engine to do stricter checks on this, like for a testing system. Does it make sense? - Well, I let it to you to decide whether it make sense to have two syntaxes which look the same but do very different things (being ignored vs. throwing a terminating error) and whether it makes sense to push a system where the core language behaves differently depending on the system (is an extension using this hook loaded? which of them? - one which casts or one which throws an error?) ... I seriously hoped PHP was out of the era of introducing new inconsistencies ...<br /> 
            </div>
        </content>
        <dc:subject>php</dc:subject>
<dc:subject>php.next</dc:subject>
<dc:subject>type hints</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/149-Fuer-unsere-Sicherheit!.html" rel="alternate" title="Für unsere Sicherheit!" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2010-11-23T11:40:00Z</published>
        <updated>2010-11-23T08:51:35Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=149</wfw:comment>
    
        <slash:comments>0</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=149</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/7-Gesellschaft" label="Gesellschaft" term="Gesellschaft" />
    
        <id>http://schlueters.de/blog/archives/149-guid.html</id>
        <title type="html">Für unsere Sicherheit!</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>Wurde ja langsam Zeit:</p> 
<blockquote> 
<p>Die Regierung erwägt angesichts der Terrorgefahr eine Aufrüstung der 
Geheimdienste. Der Bund Deutscher Kriminalbeamter fordert 
Bundeswehr-Unterstützung im Inneren.</p> 
<p> (<a href="http://www.zeit.de/politik/deutschland/2010-11/geheimdienste-organisation">Quelle</a>)</p> 
</blockquote> 
<p>Dann können die Leute mit den Maschinenpistolen am Bahnhof wenigstens damit umgehen, wenn da wild in die Menge geschossen wird.</p> 
<blockquote> 
<p>Der Einsatz dieser Waffen erlangt seine Bedeutung vor allem gegen 
Gruppenziele auf kurze Distanz, da eine hohe Feuerdichte erreicht wird</p> 
<p>(<a href="http://de.wikipedia.org/wiki/Maschinenpistole">Quelle</a>)</p> 
</blockquote> 
<p>hachja ... <a href="http://www.tagesschau.de/inland/terrorwarnung156.html">Propagandadrohungen</a> sind ja was tolles!<br /></p> 
            </div>
        </content>
        <dc:subject>bundesregierung</dc:subject>
<dc:subject>bundeswehr</dc:subject>
<dc:subject>geheimdienste</dc:subject>
<dc:subject>panik</dc:subject>
<dc:subject>politik</dc:subject>
<dc:subject>regierung</dc:subject>
<dc:subject>terroristen</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/147-Changes-in-PHP-trunk-No-more-extension-for-sqlite-version-2.html" rel="alternate" title="Changes in PHP trunk: No more extension for sqlite version 2" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2010-11-22T06:48:00Z</published>
        <updated>2010-11-22T10:13:07Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=147</wfw:comment>
    
        <slash:comments>0</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=147</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/147-guid.html</id>
        <title type="html">Changes in PHP trunk: No more extension for sqlite version 2</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>I plan to continue <a href="http://schlueters.de/blog/plugin/tag/php%FFnext">my series about new stuff in PHP's trunk</a> but for now just a short note about something which was removed: PHP 5.3 has different ways to access SQLite databases of all kinds. Two of them are provided by the <em>sqlite</em> extension: The <a href="http://php.net/manual/en/ref.sqlite.php"><em>sqlite_</em> group of functions</a> and the <a href="http://php.net/manual/en/ref.pdo-sqlite.php"><em>pdo_sqlite2</em> driver</a>. The issue there is that this depends on the SQLite 2 library which isn't supported by upstream anymore for a few years. It was a logical step therefore to remove this extension from PHP trunk. The support for the <a href="http://php.net/sqlite3">sqlite3 extension </a>and the <a href="http://php.net/manual/en/ref.pdo-sqlite.php">PDO_sqlite driver</a> (same link as above, read it carefully), which use version 3 of the library, are continued. Please note that the disk format changed from version 2 to 3 so you might not only have to change the application but also to recreate the database file. This change will most likely appear in the feature version of PHP, which will most likely be called PHP 5.4.<br /></p> 
            </div>
        </content>
        <dc:subject>php</dc:subject>
<dc:subject>php extensions</dc:subject>
<dc:subject>php.next</dc:subject>
<dc:subject>sqlite</dc:subject>
<dc:subject>sqlite3</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/146-mysqlnd-plugins-for-PHP-in-practice.html" rel="alternate" title="mysqlnd plugins for PHP in practice" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2010-11-06T10:21:00Z</published>
        <updated>2010-11-09T16:22:27Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=146</wfw:comment>
    
        <slash:comments>3</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=146</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/146-guid.html</id>
        <title type="html">mysqlnd plugins for PHP in practice</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                <p>If you follow my <a href="http://schlueters.de/blog/archives/145-Slides-from-IPC-and-PHP-Barcelona.html">blog</a> or <a href="http://twitter.com/phperror/status/29063856120">twitter</a> stream you might know I've recently been at Barcelona to attend the PHP Barcelona conference. Conferences are great for exchanging ideas one of the ideas I discussed with <a href="http://www.combell.com/">Combell</a>'s Thijs Feryn: They are a hosting company providing managed MySQL instances to their customers, as such they run multiple MySQL servers and each server serves a few of their customers. Now they have to provide every customer with database credentials, including a host name to connect to. The issue there is that a fixed hostname takes flexibility out of the setup. Say you have db1.example.com and db2.example.com over time you figure out that there are two high load customers on db1 while db2 is mostly idle. You might want to move the data from one customer over to db2 to share the load. This means you have to ask the customer to change his application configuration at the time you're moving the data. Quite annoying task.</p> 
<p>Now there's a solution: MySQL Proxy. The proxy is a daemon sitting in between of the application/web servers and MySQL something like in the picture below.<br /></p> 
<p><!-- s9ymdb:60 --><img width="521" height="332" src="http://schlueters.de/blog/uploads/code/mysql_proxy_in_between.png" class="serendipity_image_center" /><br /></p> 
<p>The proxy can be scripted using lua so it is not too hard to implement a feature which chooses the database server to actually connect to. The customer is then told to connect to the proxy and depending on the username given he is redirected to a specific system. All magic happens transparent in the background. This is nice but not without issues: There is one more daemon to monitor, the proxy sitting in between adds latency, and so on.</p> 
<p>In case you attended a recent talk by Ulf or me you certainly learned about mysqlnd plugins. We always compare mysqlnd plugins with the MySQL Proxy, so let's take a closer look: The plugins are PHP extensions, usually written in C, hooking into mysqlnd, the native driver for PHP, overriding parts of mysqlnd's internals. mysqlnd, introduced in PHP 5.3, is the implementation of the MySQL Client-Server-Protocol sitting invisible below the PHP extensions ext/mysql, mysqli and PDO_mysql. This means any plugin to mysqlnd can transparently change the behavior without an changes to the actual application.<br /></p> 
<p> 
<iframe width="670" height="550" src="/presentations/#p=20101030_barcelona_MySQL_Caching&amp;n=5"></iframe> 
</p> 
<p>Now with this plugin facility we can move the code for the server selection from the proxy directly in PHP. By doing this we will have almost no overhead and due to the deep integration less work for monitoring and no additional fault component.<br /></p> 
<p><!-- s9ymdb:61 --><img width="521" height="339" src="http://schlueters.de/blog/uploads/code/mysql_proxy_as_plugin.png" class="serendipity_image_center" /><br /></p> 
<p>So let's look in the implementation of such a simple plugin: The goal is having an extension which overrides the server name given by the user by one set in&#160; a special configuration file so the user is transparently redirected. The configuration file format used is a INI file. As said above a mysqlnd plugin is a regular PHP extension, even though we usually won't export functions to PHP userland. A quick note before we really start: I won't discuss all parts of the PHP API in detail, please see the resources linked below for more on that. <br /></p> 
<p>The first thing PHP looks at while loading an extension is a module entry. In our case there is one special thing: We add a dependency to mysqlnd, to make sure mysqlnd was initialised before this extension is initialised. You can also see that I have chosen the name <em>mysqlnd_server_locator</em>.</p> 
<blockquote> 
<pre>static const zend_module_dep mysqlnd_server_locator_deps[] = {
    ZEND_MOD_REQUIRED("mysqlnd")
    {NULL, NULL, NULL}
};

zend_module_entry mysqlnd_server_locator_module_entry = {
    STANDARD_MODULE_HEADER_EX,
    NULL,
    mysqlnd_server_locator_deps,
    "mysqlnd_server_locator",
    NULL,
    PHP_MINIT(mysqlnd_server_locator),
    PHP_MSHUTDOWN(mysqlnd_server_locator),
    NULL,
    NULL,
    NULL,
    "0.1",
    STANDARD_MODULE_PROPERTIES
};
</pre> 
</blockquote> 
<p>On PHP startup the module initializer, MINIT, is being called. We want to override the connect method from mysqlnd's connection related functions. Additionally I initialize a HashTable which will hold the translation table.</p> 
<blockquote> 
<pre>static int plugin_id;
static func_mysqlnd_conn__connect orig_mysqlnd_conn_connect_method;

static HashTable server_list;
static int server_list_init = 0;

PHP_MINIT_FUNCTION(mysqlnd_server_locator)
{
    struct st_mysqlnd_conn_methods *conn_methods;

    plugin_id = mysqlnd_plugin_register();
    conn_methods = mysqlnd_conn_get_methods();

    orig_mysqlnd_conn_connect_method = conn_methods-&gt;connect;
    conn_methods-&gt;connect = MYSQLND_METHOD(mysqlnd_server_locator, connect);

    if (zend_hash_init(&amp;server_list, 10, NULL, free, 1) == FAILURE) {
        php_error_docref(NULL TSRMLS_CC, E_WARNING, "Failed to init server_list table");
        return FAILURE;
    }

    return SUCCESS;
}
</pre> 
</blockquote> 
<p>One thing to note here is that I don't actually load the translation table, yet. This is due to issues I had while using the ini scanner during PHP's initialization phase and having the mechanism to load it later has the benefit of being ale to update the table without having to restart PHP. Anyways the above function should be relatively clear. We tell mysqlnd that a plugin is around, store the connection method pointer in a safe place and set our own connection method and then init the HashTable.</p> 
<p>During PHP shutdown we will free this table again:</p> 
<blockquote> 
<pre>PHP_MSHUTDOWN_FUNCTION(mysqlnd_server_locator)
{
    zend_hash_destroy(&amp;server_list);
    return SUCCESS;
}

</pre> 
</blockquote> 
<p>Now let's look at the implementation of the overridden connect method. At first this looks complex as it takes tons of parameters but we simply pass them through and don't have to care about them. All we care about are two things: Firstly we make sure the the translation table was initilised, then we look for the username in the table, if the user exists in the table we take the hostname given in the table, else we connect to the host requested by the user.</p> 
<blockquote> 
<pre>static enum_func_status MYSQLND_METHOD(mysqlnd_server_locator, connect)(MYSQLND * conn,
        const char *host, const char *user,
        const char *passwd, unsigned int passwd_len,
        const char *db, unsigned int db_len,
        unsigned int port,
        const char * socket_or_pipe,
        unsigned int mysql_flags
        TSRMLS_DC)
{
    char **new_host;
    char *actual_host = host;

    if (!server_list_init) {
        mysqlnd_server_locator_init_server_list(TSRMLS_C);
        server_list_init = 1;
    }

    if (zend_hash_find(&amp;server_list, user, strlen(user) + 1, (void**)&amp;new_host) == SUCCESS) {
        actual_host = *new_host;
    }

    return orig_mysqlnd_conn_connect_method(conn, actual_host, user, passwd, passwd_len, db, db_len, port, socket_or_pipe, mysql_flags TSRMLS_CC);
}

</pre> 
</blockquote> 
<p>Please note that this method is not thread-safe and should, in this form, only be used in non-threaded environments. This is fixed in a version linked below, which also does one more thing: It will always check whether the ini file was modified since we read it, but let's keep it simple here. As said the configuration is a ini file which simply consists of username=host pairs:</p> 
<blockquote> 
<pre>johannes=db1.example.com
guybrush=db1.example.com
sam=db2.example.com
max=db2.example.com
bernard=db1.example.com
</pre> 
</blockquote> 
<p>Such files can be parsed by PHP, I won't go into the details of the implementation here.</p> 
<blockquote> 
<pre>static void mysqlnd_server_locator_ini_parser_cb(zval *arg1, zval *arg2, zval *arg3, int callback_type, void *list_v TSRMLS_DC)
{
    HashTable *list = (HashTable*)list_v;
    char *hostname;
    
    if (!arg1 || !arg2) {
        return; 
    }
    
    switch (callback_type)
    {
        case ZEND_INI_PARSER_ENTRY:
            hostname = pestrndup(Z_STRVAL_P(arg2), Z_STRLEN_P(arg2), 1);
            zend_hash_update(list, Z_STRVAL_P(arg1), Z_STRLEN_P(arg1) + 1, &amp;hostname, sizeof(char *), NULL);
            break;
        case ZEND_INI_PARSER_SECTION:
            break;
        case ZEND_INI_PARSER_POP_ENTRY:
            php_error_docref(NULL TSRMLS_CC, E_NOTICE, "Array syntax not allowed in ini file");
            break;
        default:
            php_error_docref(NULL TSRMLS_CC, E_NOTICE, "Unexpected callback_type while parsing server list ini file");
            break; 
    }   
}       

static int mysqlnd_server_locator_init_server_list(TSRMLS_D)
{
    zend_file_handle fh;

    memset(&amp;fh, 0, sizeof(fh));
    fh.filename = "/tmp/server.ini";
    fh.type = ZEND_HANDLE_FILENAME;

    if (zend_parse_ini_file(&amp;fh, 0, ZEND_INI_SCANNER_NORMAL, mysqlnd_server_locator_ini_parser_cb, &amp;server_list TSRMLS_CC) == FAILURE) {
        php_error_docref(NULL TSRMLS_CC, E_WARNING, "Failed to parse server list ini file");
        return FAILURE;
    }

    return SUCCESS;
}
</pre> 
</blockquote> 
<p>And that's it. Now let's have a look at some PHP code running while this extension is loaded:</p> 
<blockquote>$ php -r 'mysql_connect(&quot;loalhost&quot;, &quot;johannes&quot;, &quot;supersecretpasswordforthis&quot;);'<br />Warning: mysql_connect(): php_network_getaddresses: getaddrinfo failed: node name or<br />service name not known in Command line code on line 1<br /><br />Warning: mysql_connect(): [2002] php_network_getaddresses: getaddrinfo failed: node<br />name or servi (trying to connect via tcp://db1.example.com:3306) in Command line code on line 1<br /></blockquote> 
<p>Neat, isn't it? - I also packaged this code in an slightly improved version. <a href="http://schlueters.de/blog/uploads/code/mysqlnd_server_locator.tar.bz2">This version</a> uses a php.ini setting for configuring the location of the extension's ini file, solves the threading issue mentioned above and automatically reloads the configuration file in case it was changed. Note that this code comes for educational purpose as-is only and I take no responsibility of any form.</p> 
<p>This won't solve all problem's in the case of Combell as they want to provide external access or access from other applications, too. But I could imagine a solution using such a plugin for PHP as the overhead is minimal (in the version above one hash lookup, in the download version one hash lookup and a, well cached, stat call during connect which both can be neglected) and a proxy-based solution for other systems.<br /></p> 
<p>Some more resources:</p> 
<ul> 
<li><a href="http://php.net/internals2">PHP Internals Documentation</a></li> 
<li><a href="http://php.net/mysqlnd.plugin">mysqlnd plugin API</a></li> 
<li><a href="http://blog.ulf-wendel.de/">Ulf's blog with lot's of related postings</a></li> 
<li><a href="http://www.slideshare.net/nixnutz">Ulf's presentations</a><br /></li> 
</ul> 
            </div>
        </content>
        <dc:subject>coding</dc:subject>
<dc:subject>mysql</dc:subject>
<dc:subject>mysql proxy</dc:subject>
<dc:subject>mysqlnd</dc:subject>
<dc:subject>mysqlnd plugins</dc:subject>
<dc:subject>php conferences</dc:subject>
<dc:subject>php53</dc:subject>
<dc:subject>phpbarcelona</dc:subject>

    </entry>
    <entry>
        <link href="http://schlueters.de/blog/archives/145-Slides-from-IPC-and-PHP-Barcelona.html" rel="alternate" title="Slides from IPC and PHP Barcelona" />
        <author>
            <name>Johannes Schlüter</name>
                    </author>
    
        <published>2010-10-31T22:09:55Z</published>
        <updated>2010-11-06T10:29:27Z</updated>
        <wfw:comment>http://schlueters.de/blog/wfwcomment.php?cid=145</wfw:comment>
    
        <slash:comments>1</slash:comments>
        <wfw:commentRss>http://schlueters.de/blog/rss.php?version=atom1.0&amp;type=comments&amp;cid=145</wfw:commentRss>
    
            <category scheme="http://schlueters.de/blog/categories/3-PHP" label="PHP" term="PHP" />
    
        <id>http://schlueters.de/blog/archives/145-guid.html</id>
        <title type="html">Slides from IPC and PHP Barcelona</title>
        <content type="xhtml" xml:base="http://schlueters.de/blog/">
            <div xmlns="http://www.w3.org/1999/xhtml">
                Recently I gave a few public presentations and started to convert the slides for making them available online. Here's the first bunch with slides from two conferences which were held in October, the International PHP Conference in Germany and PHP Barcelona in Spain. As always: The spoken word is missing on the slides ...<br /> <br /><a href="http://schlueters.de/blog/archives/145-Slides-from-IPC-and-PHP-Barcelona.html#extended">Continue reading "Slides from IPC and PHP Barcelona"</a>
            </div>
        </content>
        <dc:subject>conferences</dc:subject>
<dc:subject>ipc</dc:subject>
<dc:subject>ipc10</dc:subject>
<dc:subject>mysql</dc:subject>
<dc:subject>mysqlnd</dc:subject>
<dc:subject>mysqlnd plugins</dc:subject>
<dc:subject>php</dc:subject>
<dc:subject>php53</dc:subject>
<dc:subject>phpbarcelona</dc:subject>
<dc:subject>presentations</dc:subject>
<dc:subject>talks</dc:subject>

    </entry>

</feed>
