New memcached extension

» 29 January 2009 » In PHP, Work »

The first project that I’ve been working on since joining Digg has seen the light of day. It’s a new PHP extension for interfacing with memcache servers and it is based on the libmemcached library, which is becoming the standard client library for this task. It’s used by Python, Ruby, Perl, and now – PHP. The extension is available from PECL [1]. There is another memcache PECL extension, but this one offloads the intricacies of communicating with memcache onto libmemcached and instead concentrates on exposing a sensible API and some cool features like asynchronous requests and read-through caching callbacks.

I’m excited about this release and looking forward to putting out more stuff soon.

Now, to write the documentation..

1. http://pecl.php.net/package/memcached

Trackback URL

1 Comment 9 Comments 1 Tweet

51 Comments on "New memcached extension"

  1. andrei
    Robert
    29/01/2009 at 12:41 pm Permalink

    Will this help me get more MySpace friends?

  2. andrei
    Mike P
    29/01/2009 at 1:00 pm Permalink

    @Robert only if you leverage the platform to maximize synergies.

    Andrei, that’s awesome. Will have to play with this later.

  3. andrei
    Justin
    29/01/2009 at 3:09 pm Permalink

    What advantages does this have over the other extension, currently? I understand using a more shared and proven underlying library, but what’ll be different on the PHP side?

    (This is before looking at the package :-) )

  4. andrei
    Andrei
    29/01/2009 at 3:11 pm Permalink

    @Justin, a few things are different. The API is more extensive and provides for such features as server-specific hashing, read-through cache and value callbacks, asynchronous multi-gets, and other stuff. It’ll be apparent in the docs.

  5. andrei
    Justin
    29/01/2009 at 3:31 pm Permalink

    Was just looking through the lib docs as well as looking at some of your code – definitely stuff we want to take advantage of. We’ll be following this :-)

  6. andrei
    RJ
    30/01/2009 at 3:27 am Permalink

    Nice one, we’ll test this out for Last.fm too. Our hacked extension with separate consistent hashing is far from ideal.
    Thanks for publishing.

  7. andrei
    Adam Rifkin
    30/01/2009 at 3:56 pm Permalink

    Brilliant. Thanks, Andrei and Digg!

  8. andrei
    Chris Williams
    01/02/2009 at 11:58 am Permalink

    Looking forward to the docs on this! Thanks for your work :-)

  9. andrei
    Alexandr
    02/02/2009 at 4:22 am Permalink

    Andrei, do you plan to build binaries? specially for windows environments.

  10. andrei
    RJ
    02/02/2009 at 4:58 am Permalink

    Hi,
    Please see my trivial patch in this mailing list post to make this client compatible with others using serialized/compressed flags, and also for ketama-compatible hashing support.
    RJ

  11. andrei
    Andrei
    02/02/2009 at 10:42 am Permalink

    @Alexandr, I have no facilities to do this. I will have to leave it to people that run http://pecl4win.php.net/, whenever it comes back.

    @RJ, Thanks.

  12. andrei
    Paul
    02/02/2009 at 10:59 am Permalink

    github having issues?

    fatal: read error (Connection reset by peer)

    can’t download the source to zip or tar either.

  13. andrei
    Andrei
    02/02/2009 at 11:04 am Permalink

    @Paul, I think it was having issues. I would suggest downloading from http://pecl.php.net/package/memcached as that is the default distribution point.

  14. andrei
    Josh Koenig
    03/02/2009 at 7:27 pm Permalink

    Very nice! I will tool around w/this on a couple project sites. Currently having lots of fun prepping for a presentation on scalable asynchronous data applications, and being able to note PHP moving up in this regard will be a win.

    Thanks!

  15. andrei
    Maxwell Lamb
    06/02/2009 at 10:10 am Permalink

    Andrei – great stuff – have just implemented this on our eCommerce platform, and it’s immediately solved the issues we were experiencing with memcached quietly going away and returning false on critical keys.

    One thing you’ve missed – you don’t have the getStats method in your memcached-api.php file – no biggie, but had to grep through the c to find out what it was called!

  16. andrei
    Ash Searle
    06/02/2009 at 11:41 am Permalink

    Should this new extension be able to work as a PHP session handler? e.g. with php.ini settings:


    extension=memcached
    session.save_handler=memcached
    session.save_path="tcp://127.0.0.1:11211"

    I’ve been trying to get this to work without success. Using $m = new Memcached('somename'); $m->addServer('127.0.0.1', '11211'); then calling get and set manually works fine. But the session-handling aspect seems fubar as the end of every request ends with an error:

    Warning: Unknown: Failed to write session data (memcached). Please verify that the current setting of session.save_path is correct (tcp://127.0.0.1:11211) in Unknown on line 0

    Any ideas?

    (memcached is running as normal user via memcached -d -m 32 -l 127.0.0.1 -p 11211)

  17. andrei
    Andrei
    06/02/2009 at 11:45 am Permalink

    @Ash, the session save path syntax is not quite the same as the other memcache extension. Try:

    session.save_path="127.0.0.1:11211"

  18. andrei
    Ash Searle
    06/02/2009 at 12:09 pm Permalink

    That works, thanks!

    Is there any documentation for the syntax – i.e. does it handle multiple servers and optional parameters like the earlier memcache implementations?
    (e.g. save_path="127.0.0.1:11211?persistent=1&timeout=1&retry_interval=15")

  19. andrei
    Andrei
    06/02/2009 at 12:22 pm Permalink

    @Ash, no, it’s not that advanced. For now, the syntax is the default one that libmemcached parser supports: basically, a comma-separated list of hostname:port entries, where :port is optional.

  20. andrei
    Diego Sana
    26/02/2009 at 5:07 am Permalink

    Hi Andrei, great extension, thanks for sharing. But how stable is this? Are you guys using it on production @ digg?

  21. andrei
    Andrei
    26/02/2009 at 11:10 am Permalink

    @Diego, not yet, but soon.

  22. andrei
    Rich Choy
    02/04/2009 at 3:50 pm Permalink

    @Andrei:

    Howdy!

    Is there a reference on the Web that explains each connection parameter that appears after host:port? For example what exactly does “timeout=1″ mean?

  23. andrei
    Andrei
    02/04/2009 at 10:04 pm Permalink

    @Rich, which extension are you using? :) Mine doesn’t support those extra parameters, you must be talking about pecl/memcache one.

  24. andrei
    Rich Choy
    04/04/2009 at 2:51 pm Permalink

    Sorry, I didn’t RTFP (“P” being posts).

    Yes, I’m using the PECL memcache extension. I’m going to test-drive yours this week. :)

  25. andrei
    Dan
    28/07/2009 at 2:05 pm Permalink

    Andrei,

    If getMulti can return the cas hash, why is there not a casMulti function?

  26. andrei
    Andrei
    28/07/2009 at 2:30 pm Permalink

    getMulti corresponds to an API call in libmemcached, but there is no such thing for cas, so it would have to be emulated by fetching values individually.

  27. andrei
    raghu veer
    25/10/2009 at 11:38 am Permalink

    I wish to try memcached in php, and even though I had the other pecl memcache extension installed, I thought of trying memcached based on libmemcached and got that implemented in the server. The manual do not have working example to connect, and most of the examples online in different websites present examples related to pecl memcache but not pecl memcached (which is based on libmemcached). While I think, I got the concept and thinking of how to use this in my application, I wish a push to start with a set of examples can really help, many like me, I hope you will post some sooner, thank you.

    Please give examples in procedural and object oriented methods both, thank you

  28. andrei
    Frank Orson
    09/02/2010 at 3:48 pm Permalink

    Hi Andrei,

    1) Does pecl/memcached support UDP on the client? I could not find any info about this. I know that pecl/memcache 3.0.4 supports it.

    2) Does pecl/memcached have failover support in the client?

    Thanks in advance,

    –Franky

  29. andrei
    Andrei
    09/02/2010 at 5:21 pm Permalink

    Frank,

    I’m working on next version (2.0) of the memcached extension. It’ll have UDP support and replication (failover).

  30. andrei
    joseph
    09/03/2010 at 4:47 pm Permalink

    Did any failover/replication support make it into 2.0?

    I grabbed 2.0.0 from github, and it has been working nicely for the most part – until I kill one node…

  31. andrei
    Andrei
    09/03/2010 at 6:17 pm Permalink

    joseph, did you set the OPT_NUMBER_OF_REPLICAS option?

  32. andrei
    Bill Salak
    06/04/2010 at 6:09 pm Permalink

    Nicely done, completely replaces our own wrapper for PECL/Memcache which we wrote to enable read-through-cache callbacks and the interface is much cleaner to work with than PECL/Memcache.

    Does PECL/Memcached support object references like call_user_func_array ? i.e. cache_cb can be an array in the form of array(~object reference~, ~callback method~) ? If not I’d like to suggest that feature for consideration as we’ve found it very empowering. If it does have it – it would seem to be missing from the official docs.

    thanks in advance for your answer.

  33. andrei
    Andrei
    06/04/2010 at 6:16 pm Permalink

    Bill,

    Yes, the callback can be in any of the formats supported by PHP and listed here under “Callback”:

    http://php.net/callback

  34. andrei
    Bill Salak
    07/04/2010 at 3:37 pm Permalink

    Yes of course! this makes perfect sense that it would.

    I’m finding the required signature of the callback function to be a bit limiting. The callback functions need to be so specific to the use case of the callback that we’re required to write these as wrappers to more reusable functions. It would be nice to be able to return a value from the callback and to be able to pass in an array of arguments to the Memcached methods that support the read-through cache that get expanded into the callback function signature. No complaints here, just a few suggestions.

  35. andrei
    jianzi
    22/04/2010 at 8:25 am Permalink

    Hi Andrei,
    I want use Memcached client library and UDP mod to connect memcached server. but i can’t find any way to do that.

    can’t you help me or tell me some way to do that?

    thanks.

    sorry. my english is bad. -.-!!

  36. andrei
    Andrei
    25/04/2010 at 5:17 pm Permalink

    Bill,

    What do you mean by “would be nice to return a value from the callback”? Where would this value be used? I’ll see if it would be possible to support passing additional arguments to the callbacks.

  37. andrei
    Artur Bodera
    27/04/2010 at 6:05 am Permalink

    Hey Andrei.
    Why have you ignored my email?
    I’ve sent you a patch for php-memcached to enable replicas in session save handler.

    Feel free to contact me.

    A.

  38. andrei
    Ivan Jovanovic
    29/04/2010 at 3:36 am Permalink

    Hi Andrei,

    I’m trying to figure out how to use auto_eject_host feature of libmemcached lib.
    I was thinking about setting the behavior through Memcached::setOption() since I see in code that it would handle libmemcached options as default if they are not defined in the php extension.

    Do you maybe have some experience with this?
    Would that be enough, or is there some more code to be implemented in php-memcached extension for it to work?

    Cheers,
    Ivan

  39. andrei
    Bill Salak
    01/05/2010 at 10:54 am Permalink

    Andrei, thanks for the interest. The limitation I was referring to is shown in the Example 1 from http://www.php.net/manual/en/memcached.callbacks.read-through.php

    $user_id = substr($key, 5);

    would not be necessary if the signature of the callback allowed for arguments to be passed in.

    If the read-through-cache callback api simply handed off the variable values into the callback signature and accepted whatever was returned as the value to the memcached key then there would be no need to define the callback signature outside of userspace. This is what I meant by “would be nice to return a value from the callback”.

    With the API I’ve described the onus would be on the programmer to make sure the arguments passed into the callback function via the read-through-cache callback call (memcached->get) match the signature they’ve designed for the callback. The memcached object could still be passed in but the userspace code would/could ignore this as necessary if it was always passed as the last arg to the callback function autonomously.

    These are probably just my personal preferences and the extension is functionally complete as-is so I thank you again for your interest and consideration of these ideas. I have userspace code that does this for the memcache ext you can take a look at if my explanation isn’t clear. Feedback and questions are welcome.

  40. andrei
    Andrei
    01/05/2010 at 11:12 am Permalink

    Bill,

    I think this is certainly possible. The only issue is how to have the callback indicate that the read-through (default) value for the key is not found/available. Right not this is done by returning true/false, but if we change it to return the actual key value we’d need some other way to indicate it. Plus, doing so would break BC with the scripts that already use read-through callbacks. I’m inclined to keep the current ($key, &$value, $memc) signature and simply append to it any additional values passed at the call time.

  41. andrei
    predcasna ejakulacia
    28/11/2011 at 5:09 pm Permalink

    Hey, i have a problem with this:

    Unknown: Failed to write session data (memcache).

    what does it mean? what do i have to do? im scared

  42. andrei
    Russell Royston
    21/02/2012 at 4:50 am Permalink

    Appreciate you sharing, great blog post. Cool.

  43. andrei
    Lesmeister
    14/10/2014 at 9:11 am Permalink

    Sustain the good work and generating the crowd!

  44. andrei
    smartj
    29/01/2009 at 3:17 pm Permalink

    Are their any slide decks out there for seeing how to take advantage of the new features?

    This comment was originally posted on Digg

  45. andrei
    tzangms
    26/02/2009 at 6:30 pm Permalink

    Reading: “New memcached extension » Andrei Zmievski” ( http://tinyurl.com/d5ojmh )

    This comment was originally posted on Twitter

Trackbacks

  1. [...] New memcached extension » Andrei Zmievski [...]

  2. [...] now! A new Digg employee has wrapped libmemcached into a new extension! Finally! Make Sure You Upgrade Your Plugins200K …

  3. [...] point out that the article is now a few weeks old and doesn’t include anything about the new memcached …

  4. [...] пишет Андрей Змиевский новая версия использует библиотеку libmemcached, и, если в [...]

  5. MemCache VS MemCacheD | MaXyC.Ru 25/03/2009 at 6:49 am

    [...] пишет Андрей Змиевский новая версия использует библиотеку libmemcached, и, если в [...]

  6. [...] problem. It is often the connectors between. Review your connector implementations be they mod_jk , php-memcached , or your …

Hi Stranger, leave a comment:

ALLOWED XHTML TAGS:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Subscribe to Comments

Additional comments powered by BackType