MDL-25290 cache: Implemented Cache API aka MUC
[moodle.git] / cache / README.md
CommitLineData
8139ad13
SH
1MUC development code
2====================
3
4Congratulations you've found the MUC development code.
5This code is still very much in development and as such is not (and is know to not) function correctly or completely at the moment.
6Of course that will all be well and truly sorted out WELL before this gets integrated.
7
8Sample code snippets
9--------------------
10
11A definition:
12
13 $definitions = array(
14 'core_string' => array( // Required, unique
15 'mode' => cache_store::MODE_APPLICATION, // Required
16 'component' => 'core', // Required
17 'area' => 'string', // Required
18 'requireidentifiers' => array( // Optional
19 'lang',
20 'component'
21 ),
22 'requiredataguarantee' => false, // Optional
23 'requiremultipleidentifiers' => false, // Optional
24 'overrideclass' => null, // Optional
25 'overrideclassfile' => null, // Optional
26 'datasource' => null, // Optional
27 'datasourcefile' => null, // Optional
28 'persistent' => false, // Optional
29 'ttl' => 0, // Optional
30 'mappingsonly' => false // Optional
31 'invalidationevents' => array( // Optional
32 'contextmarkeddirty'
33 ),
34 )
35 );
36
37Getting a something from a cache using the definition:
38
39 $cache = cache::make('core', 'string');
40 if (!$component = $cache->get('component')) {
41 // get returns false if its not there and can't be loaded.
42 $component = generate_data();
43 $cache->set($component);
44 }
45
46The same thing but from using params:
47
48 $cache = cache::make_from_params(cache_store::MODE_APPLICATION, 'core', 'string');
49 if (!$component = $cache->get('component')) {
50 // get returns false if its not there and can't be loaded.
51 $component = generate_data();
52 $cache->set($component);
53 }
54
55If a data source had been specified in the definition the following would be all that was needed.
56
57 $cache = cache::make('core', 'string');
58 $component = $cache->get('component');
59
60The bits that make up the cache API
61-----------------------------------
62
63There are several parts that _**will**_ make up this solution:
64
65### Loader
66The loader is central to the whole thing.
67It is used by the end developer to get an object that handles caching.
6890% of end developers will not need to know or use anything else about the cache API.
69In order to get a loader you must use one of two static methods, make, or make_with_params.
70To the end developer interacting with the loader is simple and is dictated by the cache_loader interface.
71Internally there is lots of magic going on. The important parts to know about are:
72* There are two ways to get with a loader, the first with a definition (discussed below) the second with params. When params are used they are turned into an adhoc definition with default params.
73* A loader get passed three things when being constructed, a definition, a store, and another loader or datasource if there is either.
74* If a loader is the third arg then requests will be chained to provide redundancy.
75* If a data source is provided then requests for an item that is not cached will be passed to the data source and that will be expected to load the data. If it loads data that data is stored in each store on its way back to the user.
76* There are three core loaders. One for each application, session, and request.
77* A custom loader can be used. It will be provided by the definition (thus cannot be used with adhoc definitions) and must override the appropriate core loader
78* The loader handles ttl for stores that don't natively support ttl.
79* The application loader handles locking for stores that don't natively support locking.
80
81### Store
82The store is the bridge between the cache API and a cache solution.
83Cache store plugins exist within moodle/cache/store.
84The administrator of a site can configure multiple instances of each plugin, the configuration gets initialised as a store for the loader when required in code (during construction of the loader).
85The following points highlight things you should know about stores.
86* A cache_store interface is used to define the requirements of a store plugin.
87* The store plugin can inherit the cache_is_lockable interface to handle its own locking.
88* The store plugin can inherit the cache_is_key_aware interface to handle is own has checks.
89* Store plugins inform the cache API about the things they support. Features can be required by a definition.
90** Data guarantee - Data is guaranteed to exist in the cache once it is set there. It is never cleaned up to free space or because it has not been recently used.
91** Multiple identifiers - Rather than a single string key, the parts that make up the key are passed as an array.
92** Native TTL support - When required the store supports native ttl and doesn't require the cache API to manage ttl of things given to the store.
93
94### Definition
95_Definitions were not a part of the previous proposal._
96Definitions are cache definitions. They will be located within a new file for each component/plugin at **db/caches.php**.
97They can be used to set all of the requirements of a cache instance and are used to ensure that a cache can only be interacted with in the same way no matter where it is being used.
98It also ensure that caches are easy to use, the config is stored in the definition and the developer using the cache does not need to know anything about it.
99When getting a loader you can either provide a definition name, or a set or params.
100* If you provide a definition name then the matching definition is found and used to construct a loader for you.
101* If you provide params then an adhoc definition is created. It will have defaults and will not have any special requirements or options set.
102
103Definitions are designed to be used in situations where things are more than basic.
104
105The following settings are required for a definition:
106* name - Identifies the definition and must be unique.
107* mode - Application, session, request.
108* component - The component associated the definition is associated with.
109* area - Describes the stuff being cached.
110
111The following optional settings can also be defined:
112* requireidentifiers - Any identifiers the definition requires. Must be provided when creating the loader.
113* requiredataguarantee - If set to true then only stores that support data guarantee will be used.
114* requiremultipleidentifiers - If set to true then only stores that support multiple identifiers will be used.
115* overrideclass - If provided this class will be used for the loader. It must extend on of the core loader classes (based upon mode).
116* overrideclassfile - Included if required when using the overrideclass param.
117* datasource - If provided this class will be used as a data source for the definition. It must implement the cache_data_source interface.
118* datasourcefile - Included if required when using the datasource param.
119* persistent - If set to true the loader will be stored when first created and provided to subsequent requests. More on this later.
120* ttl - Can be used to set a ttl value for data being set for this cache.
121* mappingsonly - This definition can only be used if there is a store mapping for it. More on this later.
122* invalidationevents - An array of events that should trigger this cache to invalidate.
123
124The persist option.
125As noted the persist option causes the loader generated for this definition to be stored when first created. Subsequent requests for this definition will be given the original loader instance.
126Data passed to or retrieved from the loader and its chained loaders gets cached by the instance.
127This option should be used when you know you will require the loader several times and perhaps in different areas of code.
128Because it caches key=>value data it avoids the need to re-fetch things from stores after the first request. Its good for performance, bad for memory.
129It should be used sparingly.
130
131The mappingsonly option.
132The administrator of a site can create mappings between stores and definitions. Allowing them to designate stores for specific definitions (caches).
133Setting this option to true means that the definition can only be used if a mapping has been made for it.
134Normally if no mappings exist then the default store for the definition mode is used.
135
136### Data source
137Data sources allow cache _misses_ (requests for a key that doesn't exist) to be handled and loaded internally.
138The loader gets used as the last resort if provided and means that code using the cache doesn't need to handle the situation that information isn't cached.
139They can be specified in a cache definition and must implement the cache_data_source interface.
140
141### How it all chains together.
142Consider the following if you can:
143
144Basic request for information (no frills):
145
146 => Code calls get
147 => Loader handles get, passes the request to its store
148 <= Memcache doesn't have the data. sorry.
149 <= Loader returns the result.
150 |= Code couldn't get the data from the cache. It must generate it and then ask the loader to cache it.
151
152Advanced initial request for information not already cached (has chained stores and data source):
153
154 => Code calls get
155 => Loader handles get, passes the request to its store
156 => Memcache handles request, doesn't have it passes it to the chained store
157 => File (default store) doesn't have it requests it from the loader
158 => Data source - makes required db calls, processes information
159 ...database calls...
160 ...processing and moulding...
161 <= Data source returns the information
162 <= File caches the information on its way back through
163 <= Memcache caches the information on its way back through
164 <= Loader returns the data to the user.
165 |= Code the code now has the data.
166
167Subsequent request for information:
168
169 => Code calls get
170 => Loader handles get, passes the request to its store
171 <= Store returns the data
172 <= Loader returns the data
173 |= Code has the data
174
175Other internal magic you should be aware of
176-------------------------------------------
177The following should fill you in on a bit more of the behind the scenes stuff for the cache API.
178
179### Helper class
180There is a helper class called cache_helper which is abstract with static methods.
181This class handles much of the internal generation and initialisation requirements.
182In normal use this class will not be needed outside of the API (mostly internal use only)
183
184### Configuration
185There are two configuration classes cache_config and cache_config_writer.
186The reader class is used for every request, the writer is only used when modifying the configuration.
187Because the cache API is designed to cache database configuration and meta data it must be able to operate prior to database configuration being loaded.
188To get around this we store the configuration information in a file in the dataroot.
189The configuration file contains information on the configured store instances, definitions collected from definition files, and mappings.
190That information is stored and loaded in the same way we work with the lang string files.
191This means that we use the cache API as soon as it has been included.
192
193### Invalidation
194Cache information can be invalidated in two ways.
1951. pass a definition name and the keys to be invalidated (or none to invalidate the whole cache).
1962. pass an event and the keys to be invalidated.
197
198The first method is designed to be used when you have a single known definition you want to invalidate entries from within.
199The second method is a lot more intensive for the system. There are defined invalidation events that definitions can "subscribe" to (through the definitions invalidationevents option).
200When you invalidate by event the cache API finds all of the definitions that subscribe to the event, it then loads the stores for each of those definitions and purges the keys from each store.
201This is obviously a recursive and therefor intense process.
202
203TODO's and things still to think about
204--------------------------------------
205
206 1. Definitions don't really need/use the component/area identifiers presently. They may be useful in the future... they may not be.
207 We should consider whether we leave them, or remove them.