New memcached extension
The first project that I’ve been working on since joining Digg has seen the light of day. It’s a new PHP extension for interfacing with memcache servers and it is based on the libmemcached library, which is becoming the standard client library for this task. It’s used by Python, Ruby, Perl, and now – PHP. The extension is available from PECL [1]. There is another memcache PECL extension, but this one offloads the intricacies of communicating with memcache onto libmemcached and instead concentrates on exposing a sensible API and some cool features like asynchronous requests and read-through caching callbacks.
I’m excited about this release and looking forward to putting out more stuff soon.
Now, to write the documentation..
29/01/2009 at 12:41 pm Permalink
Will this help me get more MySpace friends?
29/01/2009 at 1:00 pm Permalink
@Robert only if you leverage the platform to maximize synergies.
Andrei, that’s awesome. Will have to play with this later.
29/01/2009 at 3:09 pm Permalink
What advantages does this have over the other extension, currently? I understand using a more shared and proven underlying library, but what’ll be different on the PHP side?
(This is before looking at the package
)
29/01/2009 at 3:11 pm Permalink
@Justin, a few things are different. The API is more extensive and provides for such features as server-specific hashing, read-through cache and value callbacks, asynchronous multi-gets, and other stuff. It’ll be apparent in the docs.
29/01/2009 at 3:31 pm Permalink
Was just looking through the lib docs as well as looking at some of your code – definitely stuff we want to take advantage of. We’ll be following this
30/01/2009 at 3:27 am Permalink
Nice one, we’ll test this out for Last.fm too. Our hacked extension with separate consistent hashing is far from ideal.
Thanks for publishing.
30/01/2009 at 3:56 pm Permalink
Brilliant. Thanks, Andrei and Digg!
01/02/2009 at 11:58 am Permalink
Looking forward to the docs on this! Thanks for your work
02/02/2009 at 4:22 am Permalink
Andrei, do you plan to build binaries? specially for windows environments.
02/02/2009 at 4:58 am Permalink
Hi,
Please see my trivial patch in this mailing list post to make this client compatible with others using serialized/compressed flags, and also for ketama-compatible hashing support.
RJ
02/02/2009 at 10:42 am Permalink
@Alexandr, I have no facilities to do this. I will have to leave it to people that run http://pecl4win.php.net/, whenever it comes back.
@RJ, Thanks.
02/02/2009 at 10:59 am Permalink
github having issues?
fatal: read error (Connection reset by peer)
can’t download the source to zip or tar either.
02/02/2009 at 11:04 am Permalink
@Paul, I think it was having issues. I would suggest downloading from http://pecl.php.net/package/memcached as that is the default distribution point.
03/02/2009 at 7:27 pm Permalink
Very nice! I will tool around w/this on a couple project sites. Currently having lots of fun prepping for a presentation on scalable asynchronous data applications, and being able to note PHP moving up in this regard will be a win.
Thanks!
06/02/2009 at 10:10 am Permalink
Andrei – great stuff – have just implemented this on our eCommerce platform, and it’s immediately solved the issues we were experiencing with memcached quietly going away and returning false on critical keys.
One thing you’ve missed – you don’t have the getStats method in your memcached-api.php file – no biggie, but had to grep through the c to find out what it was called!
06/02/2009 at 11:41 am Permalink
Should this new extension be able to work as a PHP session handler? e.g. with php.ini settings:
extension=memcached
session.save_handler=memcached
session.save_path="tcp://127.0.0.1:11211"
I’ve been trying to get this to work without success. Using
$m = new Memcached('somename'); $m->addServer('127.0.0.1', '11211');then callinggetandsetmanually works fine. But the session-handling aspect seems fubar as the end of every request ends with an error:Warning: Unknown: Failed to write session data (memcached). Please verify that the current setting of session.save_path is correct (tcp://127.0.0.1:11211) in Unknown on line 0
Any ideas?
(memcached is running as normal user via
memcached -d -m 32 -l 127.0.0.1 -p 11211)06/02/2009 at 11:45 am Permalink
@Ash, the session save path syntax is not quite the same as the other memcache extension. Try:
session.save_path="127.0.0.1:11211"06/02/2009 at 12:09 pm Permalink
That works, thanks!
Is there any documentation for the syntax – i.e. does it handle multiple servers and optional parameters like the earlier memcache implementations?
(e.g.
save_path="127.0.0.1:11211?persistent=1&timeout=1&retry_interval=15")06/02/2009 at 12:22 pm Permalink
@Ash, no, it’s not that advanced. For now, the syntax is the default one that libmemcached parser supports: basically, a comma-separated list of
hostname:portentries, where:portis optional.26/02/2009 at 5:07 am Permalink
Hi Andrei, great extension, thanks for sharing. But how stable is this? Are you guys using it on production @ digg?
26/02/2009 at 11:10 am Permalink
@Diego, not yet, but soon.
02/04/2009 at 3:50 pm Permalink
@Andrei:
Howdy!
Is there a reference on the Web that explains each connection parameter that appears after host:port? For example what exactly does “timeout=1″ mean?
02/04/2009 at 10:04 pm Permalink
@Rich, which extension are you using?
Mine doesn’t support those extra parameters, you must be talking about pecl/memcache one.
04/04/2009 at 2:51 pm Permalink
Sorry, I didn’t RTFP (“P” being posts).
Yes, I’m using the PECL memcache extension. I’m going to test-drive yours this week.
28/07/2009 at 2:05 pm Permalink
Andrei,
If getMulti can return the cas hash, why is there not a casMulti function?
28/07/2009 at 2:30 pm Permalink
getMulti corresponds to an API call in libmemcached, but there is no such thing for cas, so it would have to be emulated by fetching values individually.
25/10/2009 at 11:38 am Permalink
I wish to try memcached in php, and even though I had the other pecl memcache extension installed, I thought of trying memcached based on libmemcached and got that implemented in the server. The manual do not have working example to connect, and most of the examples online in different websites present examples related to pecl memcache but not pecl memcached (which is based on libmemcached). While I think, I got the concept and thinking of how to use this in my application, I wish a push to start with a set of examples can really help, many like me, I hope you will post some sooner, thank you.
Please give examples in procedural and object oriented methods both, thank you
09/02/2010 at 3:48 pm Permalink
Hi Andrei,
1) Does pecl/memcached support UDP on the client? I could not find any info about this. I know that pecl/memcache 3.0.4 supports it.
2) Does pecl/memcached have failover support in the client?
Thanks in advance,
–Franky
09/02/2010 at 5:21 pm Permalink
Frank,
I’m working on next version (2.0) of the memcached extension. It’ll have UDP support and replication (failover).
09/03/2010 at 4:47 pm Permalink
Did any failover/replication support make it into 2.0?
I grabbed 2.0.0 from github, and it has been working nicely for the most part – until I kill one node…
09/03/2010 at 6:17 pm Permalink
joseph, did you set the OPT_NUMBER_OF_REPLICAS option?
06/04/2010 at 6:09 pm Permalink
Nicely done, completely replaces our own wrapper for PECL/Memcache which we wrote to enable read-through-cache callbacks and the interface is much cleaner to work with than PECL/Memcache.
Does PECL/Memcached support object references like call_user_func_array ? i.e. cache_cb can be an array in the form of array(~object reference~, ~callback method~) ? If not I’d like to suggest that feature for consideration as we’ve found it very empowering. If it does have it – it would seem to be missing from the official docs.
thanks in advance for your answer.
06/04/2010 at 6:16 pm Permalink
Bill,
Yes, the callback can be in any of the formats supported by PHP and listed here under “Callback”:
http://php.net/callback
07/04/2010 at 3:37 pm Permalink
Yes of course! this makes perfect sense that it would.
I’m finding the required signature of the callback function to be a bit limiting. The callback functions need to be so specific to the use case of the callback that we’re required to write these as wrappers to more reusable functions. It would be nice to be able to return a value from the callback and to be able to pass in an array of arguments to the Memcached methods that support the read-through cache that get expanded into the callback function signature. No complaints here, just a few suggestions.
22/04/2010 at 8:25 am Permalink
Hi Andrei,
I want use Memcached client library and UDP mod to connect memcached server. but i can’t find any way to do that.
can’t you help me or tell me some way to do that?
thanks.
sorry. my english is bad. -.-!!
25/04/2010 at 5:17 pm Permalink
Bill,
What do you mean by “would be nice to return a value from the callback”? Where would this value be used? I’ll see if it would be possible to support passing additional arguments to the callbacks.
27/04/2010 at 6:05 am Permalink
Hey Andrei.
Why have you ignored my email?
I’ve sent you a patch for php-memcached to enable replicas in session save handler.
Feel free to contact me.
A.
29/04/2010 at 3:36 am Permalink
Hi Andrei,
I’m trying to figure out how to use auto_eject_host feature of libmemcached lib.
I was thinking about setting the behavior through Memcached::setOption() since I see in code that it would handle libmemcached options as default if they are not defined in the php extension.
Do you maybe have some experience with this?
Would that be enough, or is there some more code to be implemented in php-memcached extension for it to work?
Cheers,
Ivan
01/05/2010 at 10:54 am Permalink
Andrei, thanks for the interest. The limitation I was referring to is shown in the Example 1 from http://www.php.net/manual/en/memcached.callbacks.read-through.php
$user_id = substr($key, 5);
would not be necessary if the signature of the callback allowed for arguments to be passed in.
If the read-through-cache callback api simply handed off the variable values into the callback signature and accepted whatever was returned as the value to the memcached key then there would be no need to define the callback signature outside of userspace. This is what I meant by “would be nice to return a value from the callback”.
With the API I’ve described the onus would be on the programmer to make sure the arguments passed into the callback function via the read-through-cache callback call (memcached->get) match the signature they’ve designed for the callback. The memcached object could still be passed in but the userspace code would/could ignore this as necessary if it was always passed as the last arg to the callback function autonomously.
These are probably just my personal preferences and the extension is functionally complete as-is so I thank you again for your interest and consideration of these ideas. I have userspace code that does this for the memcache ext you can take a look at if my explanation isn’t clear. Feedback and questions are welcome.
01/05/2010 at 11:12 am Permalink
Bill,
I think this is certainly possible. The only issue is how to have the callback indicate that the read-through (default) value for the key is not found/available. Right not this is done by returning true/false, but if we change it to return the actual key value we’d need some other way to indicate it. Plus, doing so would break BC with the scripts that already use read-through callbacks. I’m inclined to keep the current ($key, &$value, $memc) signature and simply append to it any additional values passed at the call time.
29/01/2009 at 3:17 pm Permalink
Are their any slide decks out there for seeing how to take advantage of the new features?
This comment was originally posted on Digg
26/02/2009 at 6:30 pm Permalink
Reading: “New memcached extension » Andrei Zmievski” ( http://tinyurl.com/d5ojmh )
This comment was originally posted on Twitter