Class MemcachedStorageService
- java.lang.Object
-
- net.shibboleth.utilities.java.support.component.AbstractInitializableComponent
-
- net.shibboleth.utilities.java.support.component.AbstractIdentifiedInitializableComponent
-
- net.shibboleth.utilities.java.support.component.AbstractIdentifiableInitializableComponent
-
- org.opensaml.storage.impl.memcached.MemcachedStorageService
-
- All Implemented Interfaces:
Component,DestructableComponent,IdentifiableComponent,IdentifiedComponent,InitializableComponent,StorageService
public class MemcachedStorageService extends AbstractIdentifiableInitializableComponent implements StorageService
Memcached storage service. The implementation of context names is based on the implementation of simulated namespaces discussed on the Memcached project site:https://code.google.com/p/memcached/wiki/NewProgrammingTricks#Namespacing
This storage service supports arbitrary-length context names and keys despite the 250-byte limit on memcached keys. Keys whose length is greater than 250 bytes are hashed using the SHA-512 algorithm and hex encoded to produce a 128-character key that is stored in memcached. Collisions are avoided irrespective of hashing by using the memcached add operation on all create operations which guarantees that an entry is created if and only if a key of the same value does not already exist. Note that context names and keys are assumed to have single-byte encodings in UTF-8 (i.e. ASCII characters) such that key lengths are equal to their size in bytes. Hashed keys naturally meet this requirement.
An optional context key-tracking feature is available to support
updateContextExpiration(String, Long). Key tracking is disabled by default, but can be enabled by setting theenableContextKeyTrackingparameter in theMemcachedStorageService(net.spy.memcached.MemcachedClient, int, boolean)constructor. While there is modest performance impact for create and delete operations, the feature limits the number of keys per context. With the default 1M memcached slab size, in the worst case 4180 keys are permitted per context. In many if not most situations the value is easily double that. The limitation can be overcome by increasing the slab size, which decreases overall cache memory consumption efficiency. When key tracking is disabled, there is no limit on the number of keys per context other than overall cache capacity.Limitations and requirements
- The memcached binary protocol is strong recommended for efficiency and full versioning support.
In particular,
deleteWithVersion(long, String, String)anddeleteWithVersion(long, Object)will throw runtime errors under the ASCII protocol. - Memcached server 1.4.14 or later MUST be used with binary protocol for proper handling of cache entry expiration values. See the 1.4.14 release notes for details.
-
-
Field Summary
Fields Modifier and Type Field Description protected static StringCTX_KEY_BLACKLIST_SUFFIXKey suffix for entry that contains a list of blacklisted (deleted) context keys.private static StringCTX_KEY_LIST_DELIMITERDelimiter of items in the context key list.protected static StringCTX_KEY_LIST_SUFFIXKey suffix for entry that contains a list of context keys.private org.slf4j.LoggerloggerLogger instance.private static intMAX_KEY_LENGTHMaximum length in bytes of memcached keys.private net.spy.memcached.MemcachedClientmemcacheClientMemcached client instance.private intoperationTimeoutMemcached asynchronous operation timeout in seconds.private MemcachedStorageCapabilitiesstorageCapabilitiesInvariant storage capabilities.private net.spy.memcached.transcoders.Transcoder<MemcachedStorageRecord<?>>storageRecordTranscoderHandles conversion ofMemcachedStorageRecordto bytes and vice versa.private net.spy.memcached.transcoders.Transcoder<String>stringTranscoderHandles conversion of strings to bytes and vice versa.private booleantrackContextKeysFlag that controls context key tracking.
-
Constructor Summary
Constructors Constructor Description MemcachedStorageService(net.spy.memcached.MemcachedClient client, int timeout)Creates a new instance.MemcachedStorageService(net.spy.memcached.MemcachedClient client, int timeout, boolean enableContextKeyTracking)Creates a new instance with optional context key tracking.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description booleancreate(Object value)booleancreate(String context, String key, String value, Long expiration)<T> booleancreate(String context, String key, T value, StorageSerializer<T> serializer, Long expiration)protected StringcreateNamespace(String context)Creates a cache-wide unique namespace for the given context name.booleandelete(Object value)booleandelete(String context, String key)voiddeleteContext(String context)booleandeleteWithVersion(long version, Object value)booleandeleteWithVersion(long version, String context, String key)protected voiddoDestroy()StorageCapabilitiesgetCapabilities()private <T> ThandleAsyncResult(net.spy.memcached.internal.OperationFuture<T> result)Handle async result.protected StringlookupNamespace(String context)Looks up the namespace for the given context name in the cache.private StringmemcachedKey(String... parts)Creates a memcached key from one or more parts.Objectread(Object value)<T> StorageRecord<T>read(String context, String key)<T> Pair<Long,StorageRecord<T>>read(String context, String key, long version)voidreap(String context)voidsetCapabilities(MemcachedStorageCapabilities capabilities)Sets the storage capabilities.booleanupdate(Object value)booleanupdate(String context, String key, String value, Long expiration)<T> booleanupdate(String context, String key, T value, StorageSerializer<T> serializer, Long expiration)voidupdateContextExpiration(String context, Long expiration)private booleanupdateContextKeyList(String suffix, String namespace, String key)Update context key list.booleanupdateExpiration(Object value)booleanupdateExpiration(String context, String key, Long expiration)LongupdateWithVersion(long version, Object value)LongupdateWithVersion(long version, String context, String key, String value, Long expiration)<T> LongupdateWithVersion(long version, String context, String key, T value, StorageSerializer<T> serializer, Long expiration)-
Methods inherited from class net.shibboleth.utilities.java.support.component.AbstractIdentifiableInitializableComponent
setId
-
Methods inherited from class net.shibboleth.utilities.java.support.component.AbstractIdentifiedInitializableComponent
doInitialize, getId
-
Methods inherited from class net.shibboleth.utilities.java.support.component.AbstractInitializableComponent
destroy, initialize, isDestroyed, isInitialized
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface net.shibboleth.utilities.java.support.component.IdentifiedComponent
getId
-
-
-
-
Field Detail
-
CTX_KEY_LIST_SUFFIX
protected static final String CTX_KEY_LIST_SUFFIX
Key suffix for entry that contains a list of context keys.- See Also:
- Constant Field Values
-
CTX_KEY_BLACKLIST_SUFFIX
protected static final String CTX_KEY_BLACKLIST_SUFFIX
Key suffix for entry that contains a list of blacklisted (deleted) context keys.- See Also:
- Constant Field Values
-
CTX_KEY_LIST_DELIMITER
private static final String CTX_KEY_LIST_DELIMITER
Delimiter of items in the context key list.- See Also:
- Constant Field Values
-
MAX_KEY_LENGTH
private static final int MAX_KEY_LENGTH
Maximum length in bytes of memcached keys.- See Also:
- Constant Field Values
-
logger
private final org.slf4j.Logger logger
Logger instance.
-
storageRecordTranscoder
private final net.spy.memcached.transcoders.Transcoder<MemcachedStorageRecord<?>> storageRecordTranscoder
Handles conversion ofMemcachedStorageRecordto bytes and vice versa.
-
stringTranscoder
private final net.spy.memcached.transcoders.Transcoder<String> stringTranscoder
Handles conversion of strings to bytes and vice versa.
-
storageCapabilities
@Nonnull private MemcachedStorageCapabilities storageCapabilities
Invariant storage capabilities.
-
memcacheClient
@Nonnull private final net.spy.memcached.MemcachedClient memcacheClient
Memcached client instance.
-
operationTimeout
@Positive private int operationTimeout
Memcached asynchronous operation timeout in seconds.
-
trackContextKeys
private boolean trackContextKeys
Flag that controls context key tracking.
-
-
Constructor Detail
-
MemcachedStorageService
public MemcachedStorageService(@Nonnull net.spy.memcached.MemcachedClient client, @Positive int timeout)Creates a new instance.- Parameters:
client- Memcached client object. The client MUST be configured to use the binary memcached protocol, i.e.BinaryConnectionFactory, in order fordeleteWithVersion(long, String, String)anddeleteWithVersion(long, Object)to work correctly. The binary protocol is recommended for efficiency as well.timeout- Memcached operation timeout in seconds.
-
MemcachedStorageService
public MemcachedStorageService(@Nonnull net.spy.memcached.MemcachedClient client, @Positive int timeout, boolean enableContextKeyTracking)Creates a new instance with optional context key tracking.- Parameters:
client- Memcached client object. The client MUST be configured to use the binary memcached protocol, i.e.BinaryConnectionFactory, in order fordeleteWithVersion(long, String, String)anddeleteWithVersion(long, Object)to work correctly. The binary protocol is recommended for efficiency as well.timeout- Memcached operation timeout in seconds.enableContextKeyTracking- True to enable context key tracking, false otherwise. NOTE this flag must be set totruein order forupdateContextExpiration(String, Long)to work. If that capability is not needed, the flag should be set tofalsefor better performance. The feature is disabled by default.
-
-
Method Detail
-
getCapabilities
@Nonnull public StorageCapabilities getCapabilities()
- Specified by:
getCapabilitiesin interfaceStorageService
-
setCapabilities
public void setCapabilities(@Nonnull MemcachedStorageCapabilities capabilities)Sets the storage capabilities. This method should be used when the default 1M slab size is changed; theMemcachedStorageCapabilities.valueSizeshould be set equal to the chosen slab size.- Parameters:
capabilities- Memcached storage capabilities.
-
create
public boolean create(@Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key, @Nonnull @NotEmpty String value, @Nullable @Positive Long expiration) throws IOException
- Specified by:
createin interfaceStorageService- Throws:
IOException
-
create
public <T> boolean create(@Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key, @Nonnull T value, @Nonnull StorageSerializer<T> serializer, @Nullable @Positive Long expiration) throws IOException
- Specified by:
createin interfaceStorageService- Throws:
IOException
-
create
public boolean create(@Nonnull Object value) throws IOException- Specified by:
createin interfaceStorageService- Throws:
IOException
-
read
public <T> StorageRecord<T> read(@Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key) throws IOException
- Specified by:
readin interfaceStorageService- Throws:
IOException
-
read
public Object read(@Nonnull Object value) throws IOException
- Specified by:
readin interfaceStorageService- Throws:
IOException
-
read
public <T> Pair<Long,StorageRecord<T>> read(@Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key, @Positive long version) throws IOException
- Specified by:
readin interfaceStorageService- Throws:
IOException
-
update
public boolean update(@Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key, @Nonnull @NotEmpty String value, @Nullable @Positive Long expiration) throws IOException
- Specified by:
updatein interfaceStorageService- Throws:
IOException
-
update
public <T> boolean update(@Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key, @Nonnull T value, @Nonnull StorageSerializer<T> serializer, @Nullable @Positive Long expiration) throws IOException
- Specified by:
updatein interfaceStorageService- Throws:
IOException
-
update
public boolean update(@Nonnull Object value) throws IOException- Specified by:
updatein interfaceStorageService- Throws:
IOException
-
updateWithVersion
public Long updateWithVersion(@Positive long version, @Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key, @Nonnull @NotEmpty String value, @Nullable @Positive Long expiration) throws IOException, VersionMismatchException
- Specified by:
updateWithVersionin interfaceStorageService- Throws:
IOExceptionVersionMismatchException
-
updateWithVersion
@Nullable public <T> Long updateWithVersion(@Positive long version, @Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key, @Nonnull T value, @Nonnull StorageSerializer<T> serializer, @Nullable @Positive Long expiration) throws IOException, VersionMismatchException
- Specified by:
updateWithVersionin interfaceStorageService- Throws:
IOExceptionVersionMismatchException
-
updateWithVersion
@Nullable public Long updateWithVersion(@Positive long version, @Nonnull Object value) throws IOException, VersionMismatchException
- Specified by:
updateWithVersionin interfaceStorageService- Throws:
IOExceptionVersionMismatchException
-
updateExpiration
public boolean updateExpiration(@Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key, @Nullable @Positive Long expiration) throws IOException
- Specified by:
updateExpirationin interfaceStorageService- Throws:
IOException
-
updateExpiration
public boolean updateExpiration(@Nonnull Object value) throws IOException- Specified by:
updateExpirationin interfaceStorageService- Throws:
IOException
-
delete
public boolean delete(@Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key) throws IOException
- Specified by:
deletein interfaceStorageService- Throws:
IOException
-
delete
public boolean delete(@Nonnull Object value) throws IOException- Specified by:
deletein interfaceStorageService- Throws:
IOException
-
deleteWithVersion
public boolean deleteWithVersion(@Positive long version, @Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key) throws IOException, VersionMismatchException
- Specified by:
deleteWithVersionin interfaceStorageService- Throws:
IOExceptionVersionMismatchException
-
deleteWithVersion
public boolean deleteWithVersion(@Positive long version, @Nonnull Object value) throws IOException, VersionMismatchException
- Specified by:
deleteWithVersionin interfaceStorageService- Throws:
IOExceptionVersionMismatchException
-
reap
public void reap(@Nonnull @NotEmpty String context) throws IOException
- Specified by:
reapin interfaceStorageService- Throws:
IOException
-
updateContextExpiration
public void updateContextExpiration(@Nonnull @NotEmpty String context, @Nullable Long expiration) throws IOException
- Specified by:
updateContextExpirationin interfaceStorageService- Throws:
IOException
-
deleteContext
public void deleteContext(@Nonnull @NotEmpty String context) throws IOException
- Specified by:
deleteContextin interfaceStorageService- Throws:
IOException
-
doDestroy
protected void doDestroy()
- Overrides:
doDestroyin classAbstractInitializableComponent
-
lookupNamespace
protected String lookupNamespace(String context) throws IOException
Looks up the namespace for the given context name in the cache.- Parameters:
context- Context name.- Returns:
- Corresponding namespace for given context or null if no namespace exists for context.
- Throws:
IOException- On memcached operation errors.
-
createNamespace
protected String createNamespace(String context) throws IOException
Creates a cache-wide unique namespace for the given context name. The context-namespace mapping is stored in the cache.- Parameters:
context- Context name.- Returns:
- Namespace name for given context.
- Throws:
IOException- On memcached operation errors.
-
memcachedKey
private String memcachedKey(String... parts)
Creates a memcached key from one or more parts.- Parameters:
parts- Key parts (i.e. namespace, local name)- Returns:
- Key comprised of 250 characters or less.
-
handleAsyncResult
private <T> T handleAsyncResult(net.spy.memcached.internal.OperationFuture<T> result) throws IOExceptionHandle async result.- Type Parameters:
T- type of result- Parameters:
result- the result- Returns:
- the result
- Throws:
IOException- if an error occurs
-
updateContextKeyList
private boolean updateContextKeyList(String suffix, String namespace, String key) throws IOException
Update context key list.- Parameters:
suffix- the suffixnamespace- the namespacekey- the storage key- Returns:
- whether the update was a success
- Throws:
IOException- if an error occurs
-
-