New memcached extension
The first project that I’ve been working on since joining Digg has seen the light of day. It’s a new PHP extension for interfacing with memcache servers and it is based on the libmemcached library, which is becoming the standard client library for this task. It’s used by Python, Ruby, Perl, and now – PHP. The extension is available from PECL [1]. There is another memcache PECL extension, but this one offloads the intricacies of communicating with memcache onto libmemcached and instead concentrates on exposing a sensible API and some cool features like asynchronous requests and read-through caching callbacks.
I’m excited about this release and looking forward to putting out more stuff soon.
Now, to write the documentation..
Will this help me get more MySpace friends?
@Robert only if you leverage the platform to maximize synergies.
Andrei, that’s awesome. Will have to play with this later.
What advantages does this have over the other extension, currently? I understand using a more shared and proven underlying library, but what’ll be different on the PHP side?
(This is before looking at the package
)
@Justin, a few things are different. The API is more extensive and provides for such features as server-specific hashing, read-through cache and value callbacks, asynchronous multi-gets, and other stuff. It’ll be apparent in the docs.
Was just looking through the lib docs as well as looking at some of your code – definitely stuff we want to take advantage of. We’ll be following this
Nice one, we’ll test this out for Last.fm too. Our hacked extension with separate consistent hashing is far from ideal.
Thanks for publishing.
Brilliant. Thanks, Andrei and Digg!
[...] New memcached extension » Andrei Zmievski [...]
[...] now! A new Digg employee has wrapped libmemcached into a new extension! Finally! Make Sure You Upgrade Your Plugins200K inserts per second? – MyISAM MySQLOpenSQL Camp Nov [...]
Looking forward to the docs on this! Thanks for your work
Andrei, do you plan to build binaries? specially for windows environments.
Hi,
Please see my trivial patch in this mailing list post to make this client compatible with others using serialized/compressed flags, and also for ketama-compatible hashing support.
RJ
@Alexandr, I have no facilities to do this. I will have to leave it to people that run http://pecl4win.php.net/, whenever it comes back.
@RJ, Thanks.
github having issues?
fatal: read error (Connection reset by peer)
can’t download the source to zip or tar either.
@Paul, I think it was having issues. I would suggest downloading from http://pecl.php.net/package/memcached as that is the default distribution point.
Very nice! I will tool around w/this on a couple project sites. Currently having lots of fun prepping for a presentation on scalable asynchronous data applications, and being able to note PHP moving up in this regard will be a win.
Thanks!
Andrei – great stuff – have just implemented this on our eCommerce platform, and it’s immediately solved the issues we were experiencing with memcached quietly going away and returning false on critical keys.
One thing you’ve missed – you don’t have the getStats method in your memcached-api.php file – no biggie, but had to grep through the c to find out what it was called!
Should this new extension be able to work as a PHP session handler? e.g. with php.ini settings:
extension=memcached
session.save_handler=memcached
session.save_path="tcp://127.0.0.1:11211"
I’ve been trying to get this to work without success. Using
$m = new Memcached('somename'); $m->addServer('127.0.0.1', '11211');then callinggetandsetmanually works fine. But the session-handling aspect seems fubar as the end of every request ends with an error:Warning: Unknown: Failed to write session data (memcached). Please verify that the current setting of session.save_path is correct (tcp://127.0.0.1:11211) in Unknown on line 0
Any ideas?
(memcached is running as normal user via
memcached -d -m 32 -l 127.0.0.1 -p 11211)@Ash, the session save path syntax is not quite the same as the other memcache extension. Try:
session.save_path="127.0.0.1:11211"That works, thanks!
Is there any documentation for the syntax – i.e. does it handle multiple servers and optional parameters like the earlier memcache implementations?
(e.g.
save_path="127.0.0.1:11211?persistent=1&timeout=1&retry_interval=15")@Ash, no, it’s not that advanced. For now, the syntax is the default one that libmemcached parser supports: basically, a comma-separated list of
hostname:portentries, where:portis optional.[...] point out that the article is now a few weeks old and doesn’t include anything about the new memcached extension recently released by Andrei Zmievski. Perhaps there could be a sequel … [...]
[...] пишет Андрей Змиевский новая версия использует библиотеку libmemcached, и, если в [...]
Hi Andrei, great extension, thanks for sharing. But how stable is this? Are you guys using it on production @ digg?
@Diego, not yet, but soon.
[...] пишет Андрей Змиевский новая версия использует библиотеку libmemcached, и, если в [...]
@Andrei:
Howdy!
Is there a reference on the Web that explains each connection parameter that appears after host:port? For example what exactly does “timeout=1″ mean?
@Rich, which extension are you using?
Mine doesn’t support those extra parameters, you must be talking about pecl/memcache one.
Sorry, I didn’t RTFP (“P” being posts).
Yes, I’m using the PECL memcache extension. I’m going to test-drive yours this week.
[...] problem. It is often the connectors between. Review your connector implementations be they mod_jk , php-memcached , or your DBCP , reviewing & adjusting these settings often yield huge [...]
Andrei,
If getMulti can return the cas hash, why is there not a casMulti function?
getMulti corresponds to an API call in libmemcached, but there is no such thing for cas, so it would have to be emulated by fetching values individually.
I wish to try memcached in php, and even though I had the other pecl memcache extension installed, I thought of trying memcached based on libmemcached and got that implemented in the server. The manual do not have working example to connect, and most of the examples online in different websites present examples related to pecl memcache but not pecl memcached (which is based on libmemcached). While I think, I got the concept and thinking of how to use this in my application, I wish a push to start with a set of examples can really help, many like me, I hope you will post some sooner, thank you.
Please give examples in procedural and object oriented methods both, thank you
Hi Andrei,
1) Does pecl/memcached support UDP on the client? I could not find any info about this. I know that pecl/memcache 3.0.4 supports it.
2) Does pecl/memcached have failover support in the client?
Thanks in advance,
–Franky
Frank,
I’m working on next version (2.0) of the memcached extension. It’ll have UDP support and replication (failover).
Did any failover/replication support make it into 2.0?
I grabbed 2.0.0 from github, and it has been working nicely for the most part – until I kill one node…
joseph, did you set the OPT_NUMBER_OF_REPLICAS option?
Nicely done, completely replaces our own wrapper for PECL/Memcache which we wrote to enable read-through-cache callbacks and the interface is much cleaner to work with than PECL/Memcache.
Does PECL/Memcached support object references like call_user_func_array ? i.e. cache_cb can be an array in the form of array(~object reference~, ~callback method~) ? If not I’d like to suggest that feature for consideration as we’ve found it very empowering. If it does have it – it would seem to be missing from the official docs.
thanks in advance for your answer.
Bill,
Yes, the callback can be in any of the formats supported by PHP and listed here under “Callback”:
http://php.net/callback
Yes of course! this makes perfect sense that it would.
I’m finding the required signature of the callback function to be a bit limiting. The callback functions need to be so specific to the use case of the callback that we’re required to write these as wrappers to more reusable functions. It would be nice to be able to return a value from the callback and to be able to pass in an array of arguments to the Memcached methods that support the read-through cache that get expanded into the callback function signature. No complaints here, just a few suggestions.
Hi Andrei,
I want use Memcached client library and UDP mod to connect memcached server. but i can’t find any way to do that.
can’t you help me or tell me some way to do that?
thanks.
sorry. my english is bad. -.-!!
Bill,
What do you mean by “would be nice to return a value from the callback”? Where would this value be used? I’ll see if it would be possible to support passing additional arguments to the callbacks.
Hey Andrei.
Why have you ignored my email?
I’ve sent you a patch for php-memcached to enable replicas in session save handler.
Feel free to contact me.
A.
Hi Andrei,
I’m trying to figure out how to use auto_eject_host feature of libmemcached lib.
I was thinking about setting the behavior through Memcached::setOption() since I see in code that it would handle libmemcached options as default if they are not defined in the php extension.
Do you maybe have some experience with this?
Would that be enough, or is there some more code to be implemented in php-memcached extension for it to work?
Cheers,
Ivan
Andrei, thanks for the interest. The limitation I was referring to is shown in the Example 1 from http://www.php.net/manual/en/memcached.callbacks.read-through.php
$user_id = substr($key, 5);
would not be necessary if the signature of the callback allowed for arguments to be passed in.
If the read-through-cache callback api simply handed off the variable values into the callback signature and accepted whatever was returned as the value to the memcached key then there would be no need to define the callback signature outside of userspace. This is what I meant by “would be nice to return a value from the callback”.
With the API I’ve described the onus would be on the programmer to make sure the arguments passed into the callback function via the read-through-cache callback call (memcached->get) match the signature they’ve designed for the callback. The memcached object could still be passed in but the userspace code would/could ignore this as necessary if it was always passed as the last arg to the callback function autonomously.
These are probably just my personal preferences and the extension is functionally complete as-is so I thank you again for your interest and consideration of these ideas. I have userspace code that does this for the memcache ext you can take a look at if my explanation isn’t clear. Feedback and questions are welcome.
Bill,
I think this is certainly possible. The only issue is how to have the callback indicate that the read-through (default) value for the key is not found/available. Right not this is done by returning true/false, but if we change it to return the actual key value we’d need some other way to indicate it. Plus, doing so would break BC with the scripts that already use read-through callbacks. I’m inclined to keep the current ($key, &$value, $memc) signature and simply append to it any additional values passed at the call time.
Hey, i have a problem with this:
Unknown: Failed to write session data (memcache).
what does it mean? what do i have to do? im scared
Appreciate you sharing, great blog post. Cool.
Are their any slide decks out there for seeing how to take advantage of the new features?
This comment was originally posted on Digg
Reading: “New memcached extension » Andrei Zmievski” ( http://tinyurl.com/d5ojmh )
This comment was originally posted on Twitter