Bullet Cache User Guide

The server binary executable is called mdcached. The most simple way to start it is without any arguments. The server in its default configuration may not be optimally configured but it is good enough for most workloads. The currently supported command line arguments are:

Most of the arguments are very workload-specific and should not be changed without a good reason and careful testing. Generally, workloads with lots of short queries will benefit from more network threads and workloads with a smaller number of complex queries will benefit from more worker threads.

Server memory usage is maintained in an amortized, statistical way, and accounting is only performed for data records, not for internal server bookkeeping structures. The default record expiry decision function expires least used records by looking at record access counts. Somewhat more performance from the cache server can be wrought out by disabling this access statistics collection by not defining the DEEP_STATS symbol while compiling Bullet Cache, in which case the older records will be expired.

Generally, caching can be used in all cases when it is more expensive to create a data object than to put or retrieve it from the cache - which covers a really wide area for its use. As beneficial as caching may be to overall application performance, there are some common problems associated with the idea: efficiency of cache operations (since the point of caching is to replace a slow operation with a fast one) and the problem of data freshness (as the data in the cache can get out of sync with the main data store).

Bullet Cache’s performance is achieved by a highly efficient multithreaded program design with careful optimizations for maximum concurrency across multiple simultaneous clients. In addition to this it offers several ways by which data records can be moved in or out of the cache. One of these is by making use of the record tags, which enable applications to issue queries which go beyond simple key-value tags. By making use of record tags, the problem of maintaining cached data freshness can be simplified through expiring (deleting) a set of records matching certain tags.

Though Bullet Cache is designed specifically as a fast and feature-full cache server, it can also be used for some tasks which are only tangentially related to this core purpose.

One of the basic uses for Bullet Cache is as a cache for application objects. There are several reasons why such caches are desirable in modern Web application development:

The usual application pattern in this case is to create classes of objects or structures carrying the often accessed data in a way which requires the least processing after the data is fetched from the cache and before it is used (either for further preparation or directly for displaying to the user), make them serializable (marshalable) and create the interface to the cache in the program location where the data is supposed to be used.

The cache interface is often inserted in the program flow so that it conditionally replaces certain modules or parts of regular processing.

Databases are the foundation of most non-trivial applications, and especially Web applications where the possibilities for client storage are very limited. Depending on application specifics and the business requirements it is supposed to implement, the database might contain a complex schema with a number of relations and even parts of the business logic - all of which bring great convenience and powerful data manipulation features to the application but at the same time mean that database queries grow complex and slow. Even if the database schema is relatively simple, the sheer number of records in the database might grow to such extent as to saturate available IO or even CPU processing capabilities of the database server.

The use of the application-level database cache allows the application full control over which data is cached and in what way. The principle is similar to that of creating an application object cache but on the level of database records, making it easier to implement in some circumstances.

The most convenient way to implement the database cache in an application is by establishing a central module (or a principle) which the application uses to access the database.

This module can be supplemented with the following functionality:

Integrating a cache on the database layer can quickly make a huge impact on the overall application performance, but care must be taken not to inadvertently cache sensitive queries, such as those used related to application security system (e.g. user permissions).

Although Bullet Cache is not intended to be a persistent database, its fast operation and scalability can be desirable features for certain applications, for example:

In general, Bullet Cache can be considered as a special, non-persistent NoSQL database, useful to some applications.

Bullet Cache operates as a network server which efficiently supports a large number of connected clients, so it can be used as a "meeting ground" in which applications share data. Some examples for such data sharing are:

When used in this way, Bullet Cache behaves as shared storage.

Among operations supported by Bullet Cache are the so-called ATOMIC operations on data records which can be used as primitives in creating a distributed lock manager. The atomic operations handle records which contain 64-bit signed integers and perform one of the following (see the ATOMIC API for details):

The primary use case for this set of operations is to support reliable counters and similar numeric data operations, but they can be used to implement synchronization functions to be used by multiple connected clients.

Another type of supported operations are the virtual tag stack (TSTACK) operations. Each record tag can be considered to classify records into virtual stacks (FIFO queues), and there are TSTACK API functions which operate on the records in the cache server as if they belong to such stacks. As there can be arbitrary many different record tags (the whole key-value pair of a tag is considered to designate a virtual stack), there can also be arbitrary many different virtual stacks.

Note that the FIFO property of virtual stacks is only incidental and the "virtual stacks" can also be considered as ordinary unordered lists (i.e. where the ordering is not important).

Bullet Cache can be used as a non-persistent (i.e. memory-only), polling message queue simply by using the virtual stack feature from multiple clients. Applications can use the TSTACK API to pass messages between themselves through the cache server. Each virtual stack can be considered a single message queue.

The structure of these messages is not in any way dictated by the cache server, they are normal opaque records stored in the cache.

Note that Bullet Cache is not a persistent database (barring periodical snapshots) and so the message queue is also not persistent. By involving the timed record expiry in the these operations, the messages can be treated as auto-expiring, but this might interfere with the FIFO nature of the queue.

An interesting use case for Bullet Cache can be found in interactive applications which may or may not be web-based. Modern web applications tend to rely on AJAX and similar technologies which enable them constant interaction with the web server backend. While the web server could with some care handle this kind of load with a large number of very small requests, it is unlikely that a heavy-weight database would handle it. Bullet Cache can help if the application is designed to use it instead of the database for the majority of the requests. On the other hand, non web-based interactive applications have a similar problem where a very large number of clients connect to a small number of servers and are constantly changing application state. Instead of storing this state in a heavy-weight database, it can be stored in Bullet Cache and then periodically transferred in bulk to the persistent storage.

The core of the Bullet Cache's data model is the key-value dictionary found in many similar servers, where both the key and the value are arbitrary binary strings (i.e. there are no constraints on their contents - they are not limited to ASCII strings and can contain \0 characters. The key-value pair with its associated metadata is called a record. The maximum length for keys is 65534 bytes (64 KiB -2) and the maximum length of values is 2147483646 (2 GiB -2). Each record has the following metadata assigned to it:

These are described in the following sections.

Record expiry time is an machine-dependent integer (time_t type) representing the classic Unix timestamp: the number of seconds since the Unix epoch. On entry into the cache server, expiry timestamps are interpreted in one of the following ways, depending on their absolute values:

On exit from the cache server (i.e. at record retrieval), the original, client-specified timestamp value is returned.

Each key-value pair can have an arbitrary number of record tags assigned to it. Record tags are themselves structured as key-value pairs, where both are 32-bit signed integers.

The tags are of deliberately constrained data type to maximize performance while at the same time offering flexibility to application authors. The interpretation of the tags is application-specific - the cache server considers them opaque.

The introduction of record tags makes the biggest difference in efficiency compared to the more common key-value only cache server. It enables applications to issue commands operating on a large number of records at once. Using records, application can retrieve or delete sets of records grouped by some criteria using the tags.

The presence of tags makes implementation of advanced functions possible, such as treating the cache server as a more complex database than a pure key-value database, or accessing the tagged records in the cache server in the form of an array or a stack.

Bullet Cache implements a deliberate consistency model in its operation. In general, all operations working on multiple records at once have at least an optional flag that makes the operations be performed atomically on all records. This means that the operations are performed on a snapshot of the cache server date at some time before the operations return their result, and independently of other operations going on in the cache server.

For example, the atomic variants of operations querying and returning a set of records always return all the records found at the start of the operation execution, while operations that would delete those records are postponed / blocked until the records are returned.

When comparing with the traditional database concurrency models, the Bullet Cache can be said to implement page-level locking with shared-exclusive locks.

The usage of shared-exclusive locks means that operations which only access the data for reading can always be concurrently executed with any other read-only operations. All operations which modify records in any way (deleting, inserting) must take an exclusive lock and thus they block concurrent access for all other operations. The granularity of concurrency in Bullet Cache is dictated by hash tables of locks which are separate for records and tags, enabling an efficient dispersal of locking operations. In the most common case (light loads) and with random record access, it is reasonable to expect that the locks protect individual records.

Note that this discussion is mostly relevant in the edge cases under high server loads, as the cache operations are very fast themselves and the locks are only acquired for very, very short times.

Though maximum record sizes constraints are relatively permissive, performance oriented applications should keep both keys and values as short as possible on general principles. As a recommendation, the keys should be limited to around 100 bytes for best efficiency.

A similar recommendation can be made about the number of distinct record tag keys in the cache server database (total across all records) be limited to around 500 for max efficiency and that the number of tags per record be limited to around a dozen.

Note that these are only general recommendations for maximizing the cache server efficiency, not precise rules and not hard limits. The client API documentation includes performance estimates which are very general and may improve in the future.

The Bullet Cache's web site contains some performance measurements.

C:

PHP:

The mc_connect_local() and mc_connect_tcp() functions connect to the Bullet Cache server and return a connection structure pointer / resource. The connection is closed and this structure / resource is freed by the mc_close_connection() function. The do_handshake argument specifies if the client library will issue a "handshake" request to the server before the functions return, establishing such details as protocol version match. This is only necessary if it is expected that the client and the server can have a version mismatch or CPU architecture mismatch.

C:

PHP:

The simple record functions put, retrieve or delete a single key-value record to or from the cache server in a simple way. The record expiry time (exptime) for mc_put_simple() can be absolute or relative.

The C API version of mc_get_simple() returns the record value in the data argument which should be freed by the caller.

C:

PHP:

Records in the Bullet Cache server also contain record tags, and there are API functions which can be used to put tagged records into the cache.

In the C API, the tags are represented by the struct mc_tag, or more commonly by a simple array of these structures. The structure is defined as:

In the PHP API, the tags are always represented by a key-value array where both the keys and the values are integers.

C:

PHP:

The main benefit of record tags is for them to be used in queries such as these. The mc_get_by_tag_values() and mc_del_by_tag_values() get or retrieve records (respectively) which are tagged with the given tag key and contain any of the given tag values. This enables application developers to efficiently retrieve or delete (expire) cached records or a certain type or belonging to a certain application-specific class (such as owned by a user, present on a web page, etc.). See use case examples.

The C functions pass the list of tag values as a simple array of integers.

The PHP functions are straightforward. The GET function returns a key-value array with the found records. The DEL function returns the number of deleted records.

C:

PHP:

As a convenience, the record’s tags can be reset to the given list by using the mc_set_tags() function. This function does not modify any other data associated with the record.

C:

PHP:

The atomic helper operations treat records specially: as 64-bit signed integers, and offer a number of "atomic" operations to be performed on them. The "atomicity" of operations simply means that the operations are strongly guaranteed to be performed on the cache server without any interference from other concurrently connected clients (and any other operations). This feature allows the creation of reliable counters and even distributed application locks. See use case examples.

The CMPSET operation can be described as: "if (value == arg1) { value = arg2; return 1; } else { return 0; }".

The ADD operation can be described as: "value += arg; return value;".

The FETCHADD operation can be described as: "tmp = value; value += arg; return tmp;".

The READANDCLEAR operation can be described as: "tmp = value; value = 0; return tmp;".

Though the C client really implements only one function from scratch (mc_atomic_op()), it is recommended that the applications use the helper aliases (mc_atomic_cmpset(), mc_atomic_add(), mc_atomic_fetchadd() and mc_atomic_readandclear()) for clarity.

The ADD and CMPSET operations can be used to create a new record for use with the atomic operations. In case of the ADD operation, the newly created record will contain the passed argument. In case of the CMPSET operation, the newly created record will contain the second argument.

All operations will fail if the record specified for the operation exists and does not contain a 64-bit quantity.

The mc_atomic_put() and mc_atomic_get() functions are auxiliary, present for conventience and ease of use. They are a simple way to create or retrieve a record for use with the atomic operations as "cooked" data, already interpreted as a signed 64-bit integer instead of being treated as a binary blob. The tags argument is optional for mc_atomic_put().

C:

PHP:

These operations retrieve, put or delete multiple records at once, which is useful for avoiding communication overhead when working with a set of records. However, the operations are slower than simple operation equivalents if they only operate on a single record.

The flags passed to can include MCMD_FLAG_ATOMICALL which indicates that the operation is to be performed atomically with regards to other operations on the server.

C:

PHP:

The record tags feature in the cache server can be used to implement virtual stacks (TSTACK). Each such virtual stack is uniquely identified by a tag (the whole key-value pair of a tag), and indeed every tagged record can be said to be inserted to virtual stacks. As such, there are no special operations which insert records to virtual stacks, but the mc_tstack_push() operation is provided as a convenience. This method accepts only the record value and its tags, leaving the record key to be auto-generated on the server and returned to the caller.

The mc_tstack_pop() operation retrieves and deletes (i.e. "pops") a record from a virtual stack identified by its tag key and tag value.

The C API function mc_tstack_pop returns the whole raw record in the mce argument, which should be freed by the caller. Helper functions can be used to inspect this record.

This section describes client / driver specific behavior important to application developers.