MDL-67264 core_course: Activity chooser new feature
[moodle.git] / cache / README.md
CommitLineData
4075433b
SH
1Moodle Universal Cache / Cache API
2==================================
8139ad13
SH
3
4Sample code snippets
5--------------------
6
7A definition:
8
9 $definitions = array(
4075433b 10 'string' => array( // Required, unique to the component
8139ad13 11 'mode' => cache_store::MODE_APPLICATION, // Required
47834bcd 12 'simplekeys' => false, // Optional
2e1e266c 13 'simpledata' => false, // Optional
8139ad13 14 'requireidentifiers' => array( // Optional
4075433b 15 'lang'
8139ad13
SH
16 ),
17 'requiredataguarantee' => false, // Optional
18 'requiremultipleidentifiers' => false, // Optional
fe352208
SH
19 'requirelockingread' => false, // Optional
20 'requirelockingwrite' => false, // Optional
dbd2ea4e 21 'requiresearchable' => false, // Optional
fe352208 22 'maxsize' => null, // Optional
8139ad13
SH
23 'overrideclass' => null, // Optional
24 'overrideclassfile' => null, // Optional
25 'datasource' => null, // Optional
26 'datasourcefile' => null, // Optional
d074fb09
SH
27 'staticacceleration' => false, // Optional
28 'staticaccelerationsize' => false, // Optional
8139ad13
SH
29 'ttl' => 0, // Optional
30 'mappingsonly' => false // Optional
31 'invalidationevents' => array( // Optional
32 'contextmarkeddirty'
33 ),
4350192a 34 'canuselocalstore' => false // Optional
46e17f04 35 'sharingoptions' => null // Optional
5f5776c1 36 'defaultsharing' => null // Optional
8139ad13
SH
37 )
38 );
39
fe352208 40Getting something from a cache using the definition:
8139ad13
SH
41
42 $cache = cache::make('core', 'string');
43 if (!$component = $cache->get('component')) {
44 // get returns false if its not there and can't be loaded.
45 $component = generate_data();
46 $cache->set($component);
47 }
48
fe352208 49The same thing but using params:
8139ad13
SH
50
51 $cache = cache::make_from_params(cache_store::MODE_APPLICATION, 'core', 'string');
52 if (!$component = $cache->get('component')) {
53 // get returns false if its not there and can't be loaded.
54 $component = generate_data();
55 $cache->set($component);
56 }
57
fe352208 58If a data source had been specified in the definition, the following would be all that was needed.
8139ad13
SH
59
60 $cache = cache::make('core', 'string');
61 $component = $cache->get('component');
62
82afd05c
SH
63Disabling the cache stores.
64There are times in code when you will want to disable the cache stores.
65While the cache API must still be functional in order for calls to it to work it is possible to disable the use of the cache stores separately so that you can be sure only the cache will function in all circumstances.
66
67 // Disable the cache store at the start of your script with:
68 define('CACHE_DISABLE_STORES', true);
69
70 // Disable the cache within your script when you want with:
71 cache_factory::disable_stores();
72 // If you disabled it using the above means you can re-enable it with:
73 cache_factory::reset();
74
3680c61a
SH
75Disabling the cache entirely.
76Like above there are times when you want the cache to avoid initialising anything it doesn't absolutely need. Things such as installation and upgrade require this functionality.
77When the cache API is disabled it is still functional however special "disabled" classes will be used instead of the regular classes that make the Cache API tick.
78These disabled classes do the least work possible and through this means we avoid all manner of intialisation and configuration.
94ef67cf 79Once disabled it cannot be re-enabled.
3680c61a
SH
80
81 // To disable the cache entirely call the following:
94ef67cf 82 define('CACHE_DISABLE_ALL', true);
82afd05c 83
fe352208
SH
84Cache API parts
85---------------
8139ad13 86
4075433b 87There are several parts that make up the Cache API.
8139ad13
SH
88
89### Loader
90The loader is central to the whole thing.
91It is used by the end developer to get an object that handles caching.
fe352208 9290% of end developers will not need to know or use anything else in the cache API.
759a3024 93In order to get a loader you must use one of two static methods, make or make_from_params.
fe352208 94The loader has been kept as simple as possible, interaction is summarised by the cache_loader interface.
8139ad13 95Internally there is lots of magic going on. The important parts to know about are:
fe352208
SH
96* There are two ways to get a loader, the first with a definition (discussed below) the second with params. When params are used they are turned into an adhoc definition with default params.
97* A loader is passed three things when being constructed, a definition, a store, and another loader or datasource if there is either.
8139ad13 98* If a loader is the third arg then requests will be chained to provide redundancy.
fe352208
SH
99* If a data source is provided then requests for an item that is not cached will be passed to the data source and that will be expected to load the data. If it loads data, that data is stored in each store on its way back to the user.
100* There are three core loaders. One for each application, session and request.
101* A custom loader can be used. It will be provided by the definition (thus cannot be used with ad hoc definitions) and must override the appropriate core loader
102* The loader handles ttl (time to live) for stores that don't natively support ttl.
8139ad13
SH
103* The application loader handles locking for stores that don't natively support locking.
104
105### Store
106The store is the bridge between the cache API and a cache solution.
107Cache store plugins exist within moodle/cache/store.
108The administrator of a site can configure multiple instances of each plugin, the configuration gets initialised as a store for the loader when required in code (during construction of the loader).
109The following points highlight things you should know about stores.
110* A cache_store interface is used to define the requirements of a store plugin.
111* The store plugin can inherit the cache_is_lockable interface to handle its own locking.
112* The store plugin can inherit the cache_is_key_aware interface to handle is own has checks.
113* Store plugins inform the cache API about the things they support. Features can be required by a definition.
67dfcb3d
MS
114 * Data guarantee - Data is guaranteed to exist in the cache once it is set there. It is never cleaned up to free space or because it has not been recently used.
115 * Multiple identifiers - Rather than a single string key, the parts that make up the key are passed as an array.
116 * Native TTL support - When required, the store supports native ttl and doesn't require the cache API to manage ttl of things given to the store.
bd188851 117* There are two reserved store names, base and dummy. These are both used internally.
8139ad13
SH
118
119### Definition
120_Definitions were not a part of the previous proposal._
121Definitions are cache definitions. They will be located within a new file for each component/plugin at **db/caches.php**.
122They can be used to set all of the requirements of a cache instance and are used to ensure that a cache can only be interacted with in the same way no matter where it is being used.
fe352208 123It also ensures that caches are easy to use, the config is stored in the definition and the developer using the cache does not need to know anything about its inner workings.
8139ad13
SH
124When getting a loader you can either provide a definition name, or a set or params.
125* If you provide a definition name then the matching definition is found and used to construct a loader for you.
fe352208 126* If you provide params then an ad hoc definition is created. It will have defaults and will not have any special requirements or options set.
8139ad13
SH
127
128Definitions are designed to be used in situations where things are more than basic.
129
130The following settings are required for a definition:
131* name - Identifies the definition and must be unique.
fe352208 132* mode - Application, session or request.
8139ad13
SH
133
134The following optional settings can also be defined:
47834bcd 135* simplekeys - Set to true if items will always and only have simple keys. Simple keys may contain a-zA-Z0-9_. If set to true we use the keys as they are without hashing them. Good for performance and possible because we know the keys are safe.
2e1e266c 136* simpledata - Set to true if you know that you will only be storing scalar values or arrays of scalar values. Avoids costly investigation of data types.
8139ad13
SH
137* requireidentifiers - Any identifiers the definition requires. Must be provided when creating the loader.
138* requiredataguarantee - If set to true then only stores that support data guarantee will be used.
139* requiremultipleidentifiers - If set to true then only stores that support multiple identifiers will be used.
fe352208
SH
140* requirelockingread - If set to true a lock will be acquired for reading. Don't use this setting unless you have a REALLY good reason to.
141* requirelockingwrite - If set to true a lock will be acquired before writing to the cache. Avoid this unless necessary.
dbd2ea4e 142* requiresearchable - If set to true only stores that support key searching will be used for this definition. Its not recommended to use this unless absolutely unavoidable.
fe352208
SH
143* maxsize - This gives a cache an indication about the maximum items it should store. Cache stores don't have to use this, it is up to them to decide if its required.
144* overrideclass - If provided this class will be used for the loader. It must extend one of the core loader classes (based upon mode).
8139ad13
SH
145* overrideclassfile - Included if required when using the overrideclass param.
146* datasource - If provided this class will be used as a data source for the definition. It must implement the cache_data_source interface.
147* datasourcefile - Included if required when using the datasource param.
d074fb09
SH
148* staticacceleration - Any data passing through the cache will be held onto to make subsequent requests for it faster.
149* staticaccelerationsize - If set to an int this will be the maximum number of items stored in the static acceleration array.
8139ad13
SH
150* ttl - Can be used to set a ttl value for data being set for this cache.
151* mappingsonly - This definition can only be used if there is a store mapping for it. More on this later.
152* invalidationevents - An array of events that should trigger this cache to invalidate.
46e17f04 153* sharingoptions - The sum of the possible sharing options that are applicable to the definition. An advanced setting.
5f5776c1 154* defaultsharing - The default sharing option to use. It's highly recommended that you don't set this unless there is a very specific reason not to use the system default.
4350192a 155* canuselocalstore - The default is to required a shared cache location for all nodes in a multi webserver environment. If the cache uses revisions and never updates key data, administrators can use a local storage cache for this cache.
fe352208 156It's important to note that internally the definition is also aware of the component. This is picked up when the definition is read, based upon the location of the caches.php file.
4075433b 157
d074fb09 158The staticacceleration option.
8139ad13 159Data passed to or retrieved from the loader and its chained loaders gets cached by the instance.
8139ad13 160Because it caches key=>value data it avoids the need to re-fetch things from stores after the first request. Its good for performance, bad for memory.
d074fb09 161Memory use can be controlled by setting the staticaccelerationsize option.
8139ad13
SH
162It should be used sparingly.
163
164The mappingsonly option.
165The administrator of a site can create mappings between stores and definitions. Allowing them to designate stores for specific definitions (caches).
166Setting this option to true means that the definition can only be used if a mapping has been made for it.
167Normally if no mappings exist then the default store for the definition mode is used.
168
46e17f04
SH
169Sharing options.
170This controls the options available to the user when configuring the sharing of a definitions cached data.
171By default all sharing options are available to select. This particular option allows the developer to limit the options available to the admin configuring the cache.
172
8139ad13
SH
173### Data source
174Data sources allow cache _misses_ (requests for a key that doesn't exist) to be handled and loaded internally.
175The loader gets used as the last resort if provided and means that code using the cache doesn't need to handle the situation that information isn't cached.
176They can be specified in a cache definition and must implement the cache_data_source interface.
177
178### How it all chains together.
fe352208 179Consider the following:
8139ad13
SH
180
181Basic request for information (no frills):
182
183 => Code calls get
184 => Loader handles get, passes the request to its store
185 <= Memcache doesn't have the data. sorry.
186 <= Loader returns the result.
187 |= Code couldn't get the data from the cache. It must generate it and then ask the loader to cache it.
188
189Advanced initial request for information not already cached (has chained stores and data source):
190
191 => Code calls get
192 => Loader handles get, passes the request to its store
193 => Memcache handles request, doesn't have it passes it to the chained store
194 => File (default store) doesn't have it requests it from the loader
195 => Data source - makes required db calls, processes information
196 ...database calls...
197 ...processing and moulding...
198 <= Data source returns the information
199 <= File caches the information on its way back through
200 <= Memcache caches the information on its way back through
201 <= Loader returns the data to the user.
202 |= Code the code now has the data.
203
204Subsequent request for information:
205
206 => Code calls get
207 => Loader handles get, passes the request to its store
208 <= Store returns the data
209 <= Loader returns the data
210 |= Code has the data
211
212Other internal magic you should be aware of
213-------------------------------------------
fe352208 214The following should fill you in on a bit more of the behind-the-scenes stuff for the cache API.
8139ad13
SH
215
216### Helper class
217There is a helper class called cache_helper which is abstract with static methods.
218This class handles much of the internal generation and initialisation requirements.
219In normal use this class will not be needed outside of the API (mostly internal use only)
220
221### Configuration
222There are two configuration classes cache_config and cache_config_writer.
223The reader class is used for every request, the writer is only used when modifying the configuration.
224Because the cache API is designed to cache database configuration and meta data it must be able to operate prior to database configuration being loaded.
225To get around this we store the configuration information in a file in the dataroot.
226The configuration file contains information on the configured store instances, definitions collected from definition files, and mappings.
227That information is stored and loaded in the same way we work with the lang string files.
228This means that we use the cache API as soon as it has been included.
229
230### Invalidation
231Cache information can be invalidated in two ways.
2321. pass a definition name and the keys to be invalidated (or none to invalidate the whole cache).
2332. pass an event and the keys to be invalidated.
234
fe352208 235The first method is designed to be used when you have a single known definition you want to invalidate entries within.
8139ad13
SH
236The second method is a lot more intensive for the system. There are defined invalidation events that definitions can "subscribe" to (through the definitions invalidationevents option).
237When you invalidate by event the cache API finds all of the definitions that subscribe to the event, it then loads the stores for each of those definitions and purges the keys from each store.
759a3024 238This is obviously a recursive, and therefore, intense process.
11cfc0f9 239
45a8db7f
SH
240### Testing
241Both the cache API and the cache stores have tests.
242Please be aware that several of the cache stores require configuration in order to be able operate in the tests.
243Tests for stores requiring configuration that haven't been configured will be skipped.
11cfc0f9 244All configuration is done in your sites config.php through definitions.
45a8db7f 245The following snippet illustrates how to configure the three core cache stores that require configuration.
11cfc0f9
SH
246
247 define('TEST_CACHESTORE_MEMCACHE_TESTSERVERS', '127.0.0.1:11211');
248 define('TEST_CACHESTORE_MEMCACHED_TESTSERVERS', '127.0.0.1:11211');
dbd2ea4e 249 define('TEST_CACHESTORE_MONGODB_TESTSERVER', 'mongodb://localhost:27017');
eefb680d 250
45a8db7f 251As of Moodle 2.8 it is also possible to set the default cache stores used when running tests.
eefb680d
SH
252You can do this by adding the following define to your config.php file:
253
45a8db7f 254 // xxx is one of Memcache, Memcached, mongodb or other cachestore with a test define.
eefb680d
SH
255 define('TEST_CACHE_USING_APPLICATION_STORE', 'xxx');
256
257This allows you to run tests against a defined test store. It uses the defined value to identify a store to test against with a matching TEST_CACHESTORE define.
45a8db7f 258Alternatively you can also run tests against an actual cache config.
eefb680d
SH
259To do this you must add the following to your config.php file:
260
261 define('TEST_CACHE_USING_ALT_CACHE_CONFIG_PATH', true');
262 $CFG->altcacheconfigpath = '/a/temp/directory/yoursite.php'
263
45a8db7f 264This tells Moodle to use the config at $CFG->altcacheconfigpath when running tests.
eefb680d 265There are a couple of considerations to using this method:
45a8db7f 266* By setting $CFG->altcacheconfigpath your site will store the cache config in the specified path, not just the test cache config but your site config as well.
eefb680d
SH
267* If you have configured your cache before setting $CFG->altcacheconfigpath you will need to copy it from moodledata/muc/config.php to the destination you specified.
268* This allows you to share a cache config between sites.
45a8db7f 269* It also allows you to use tests to test your sites cache config.
2c0eb5e6
SH
270
271Please be aware that if you are using Memcache or Memcached it is recommended to use dedicated Memcached servers.
272When caches get purged the memcached servers you have configured get purged, any data stored within them whether it belongs to Moodle or not will be removed.
4350192a 273If you are using Memcached for sessions as well as caching/testing and caches get purged your sessions will be removed prematurely and users will be need to start again.