176 lines
6.8 KiB
HTML
176 lines
6.8 KiB
HTML
|
<html>
|
||
|
|
<head>
|
||
|
|
<link rel="stylesheet" href="css/prettify.css" type="text/css" media="all"/>
|
||
|
|
<script type="text/javascript" src="js/prettify.js"></script>
|
||
|
|
|
||
|
|
<style>
|
||
|
|
body { background-color:#fff; color: #555; font-family:Verdana; font-size:14px; }
|
||
|
|
h1, h2, h3 { color:#333; font-family:Trebuchet MS}
|
||
|
|
a:link, a:visited { text-decoration:none; color:#a00; }
|
||
|
|
a:hover { text-decoration:none; color:#e00; }
|
||
|
|
|
||
|
|
.container { width:700px; margin:auto; font-size:90%;}
|
||
|
|
.comment { color:#070; }
|
||
|
|
pre.prettyprint { border:1px solid #ddd;}
|
||
|
|
</style>
|
||
|
|
|
||
|
|
</head>
|
||
|
|
<body onload="prettyPrint()">
|
||
|
|
<div class="container" id="page">
|
||
|
|
<h1>Generic cache system</h1>
|
||
|
|
<h2>Overview</h2>
|
||
|
|
<p>
|
||
|
|
GCache is a generic multilevel priority based cache system which is useful for allocating resources
|
||
|
|
across RAM, GPU, disk, network etc.</p>
|
||
|
|
|
||
|
|
<p><a href="html/index.html">Class documentation</a> created using Doxyigen is available in the <em>docs/html/</em> directory</p>
|
||
|
|
<p>A quick example might better explain the concept; see <a href="#example">a more detailed example</a>
|
||
|
|
for source code and in depth comments. we want to implement a priority cache for our
|
||
|
|
database of 10000 images. We will use 2 levels of caching, RAM and GPU while the images will be stored on disk.</p>
|
||
|
|
|
||
|
|
<ul>
|
||
|
|
<li>Each image is managed through a <em>token</em> which holds the pointer to the data and is used to assign a priority
|
||
|
|
to the image.</li>
|
||
|
|
<li>We subclass <em>Cache</em> to our RamCache and GpuCache and define function for loading from disk and
|
||
|
|
uploading the texture to the GPU</li>
|
||
|
|
<li>We can set priorities for the images according to our strategy.</li>
|
||
|
|
<li>Before rendering the image we lock the token and unlock it when done</li>
|
||
|
|
</ul>
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
<h2>Design</h2>
|
||
|
|
<p>GCache is designed for very frequent priority updates where the majority of
|
||
|
|
resurces are involved. All the system is designed to minimize the computations
|
||
|
|
required to change priorities and sorting them.</p>
|
||
|
|
|
||
|
|
<div style="text-align:center; margin-bottom:20px;">
|
||
|
|
<img src="img/architecture.png">
|
||
|
|
</div>
|
||
|
|
|
||
|
|
<h3>Token</h3>
|
||
|
|
<p>Each resource is managed through a pointer to a subclass of <code>Token<Priority></code>.
|
||
|
|
Token pointers are transferred between caches but never created or deleted by the cache system.
|
||
|
|
The user is responsible for storage of the tokens. </p>
|
||
|
|
<p>
|
||
|
|
Adding tokens to the cache is done using the function <code>Controller::addToken(Token *)</code>;
|
||
|
|
the tokens can be scheduled for removal using the function <code>Token::remove()</code> or be dropped
|
||
|
|
when the max number of tokens in the controller is reached
|
||
|
|
as set by <code>Controller::setMaxTokens(int)</code>. Use <code>Token::isInCache()</code> to check
|
||
|
|
for actual removal.</p>
|
||
|
|
|
||
|
|
|
||
|
|
<h3>Priority</h3>
|
||
|
|
<p>The <code>Token</code> class is templated over a <code>typename Priority</code> which usually
|
||
|
|
it is a floating point number, but it can be anything sortable through <code>Priority::operator<(const Priority &)</code>.
|
||
|
|
The goal of the cache system is to juggle resources around so to have the highest priority tokens in the higher cache.</p>
|
||
|
|
|
||
|
|
<p>Use function <code>Token::setPriority(Priority)</code> to set tokens priority; it does not require mutexing and it is
|
||
|
|
therefore vert fast. The change will take effect only upon a call to <code>Controller::updatePriorities()</code>.</p>
|
||
|
|
|
||
|
|
<p>Each cache holds a double heap of Tokens (see architecture picture), 'sorted' accordingly to priority. A double heap is a data structure with similar
|
||
|
|
properties to a heap but which allows quick extraction of both min and max element.</p>
|
||
|
|
|
||
|
|
<p>Priorities are sorted with a lazy approach, only when needed (usually when checking if a transfer should be performed. This
|
||
|
|
results in the higher, smaller caches being sorted more frequently than the larger lower caches.</p>
|
||
|
|
|
||
|
|
|
||
|
|
<h3>Caches</h3>
|
||
|
|
|
||
|
|
<p>The user needs to subclass the <code>Cache</code> class and override the virtual functions <code>get(Token *),
|
||
|
|
drop(Token *) and size(Token *)</code>. Each cache runs its own thread. Resource transfers (reading a file content, uploading a texture, etc)
|
||
|
|
are performed in that thread using blocking calls.</p>
|
||
|
|
|
||
|
|
<p>Cache are added to the controller using the function <code>Controller::addCache(Cache *)</code></p>
|
||
|
|
|
||
|
|
|
||
|
|
<p>The GCache employs a nested cache model: if a resource is present in a cache is also present in all the lower ones,
|
||
|
|
and thus each cache should have more capacity than the lower one. For sake of simplicity and performances each cache
|
||
|
|
is allowed to overflow its capacity by at most one resource. (see picture)</p>
|
||
|
|
|
||
|
|
<div style="text-align:center; margin-bottom:20px;">
|
||
|
|
<img src="img/overflow.png">
|
||
|
|
</div>
|
||
|
|
|
||
|
|
<h3>Locking</h3>
|
||
|
|
<p>Resources must be locked before use, to prevent cache threads to unload them while in use. An object can be locked
|
||
|
|
only if already loaded in the highest cache, otherwise the <code>lock()</code> function will return false.</p>
|
||
|
|
<p>Unlock allows the resource to be eventually dropped from the cache. Locks are recursive which means Tokens keep track
|
||
|
|
of how many time <code>lock()</code> have been called and release the resource only after <code>unlock()</code>
|
||
|
|
is called that many times.</p>
|
||
|
|
<p>Loking and unlocking costs basically zero (we are using <code>QAtomicInt</code>).</p>
|
||
|
|
|
||
|
|
|
||
|
|
<h2><a name="example"></a>Minimal example</h2>
|
||
|
|
<p>This is a minimal pseudo-code example, just to quickly cover the basis. For a real example, involving
|
||
|
|
also sending resources to the GPU using a share context check the <em>example/my_widget.h</em> file.
|
||
|
|
</p>
|
||
|
|
|
||
|
|
<pre class="prettyprint">
|
||
|
|
class MyToken: public Token<float> {
|
||
|
|
public:
|
||
|
|
MyData *data;
|
||
|
|
GLUint texture;
|
||
|
|
};
|
||
|
|
|
||
|
|
class RamCache: public Cache<MyToken> {
|
||
|
|
public:
|
||
|
|
//return size of object. return -1 on error
|
||
|
|
int get(MyToken *token) { fread(someting into something); return 1; }
|
||
|
|
//return size of objecy, cannot fail
|
||
|
|
int drop(MyToken *token) { delete memory; return 1; }
|
||
|
|
int size(MyToken *token) { return 1; }
|
||
|
|
};
|
||
|
|
|
||
|
|
class GpuCache: public Cache<MyToken> {
|
||
|
|
public:
|
||
|
|
int get(MyToken *token) { glCreate + glTexture;return 1; }
|
||
|
|
int drop(MyToken *token) { glDelete; return 1; }
|
||
|
|
int size(MyToken *token) { return 1; }
|
||
|
|
};
|
||
|
|
|
||
|
|
//generate tokens
|
||
|
|
vector<MyToken> tokens;
|
||
|
|
for(int i = 0; i < 10000; i++)
|
||
|
|
tokens.push_back(MyToken(i));
|
||
|
|
|
||
|
|
//create ram and gpu caches
|
||
|
|
RamCache ram;
|
||
|
|
ram.setCapacity(5000);
|
||
|
|
GpuCache gpu;
|
||
|
|
gpu.setCapacity(1000);
|
||
|
|
|
||
|
|
//create controller and add caches
|
||
|
|
Controller<MyToken> controller;
|
||
|
|
controller.addCache(&ram);
|
||
|
|
controller.addCache(&gpu);
|
||
|
|
|
||
|
|
//tell controller about tokens and start
|
||
|
|
for(int i = 0; i < tokens.size(); i++) {
|
||
|
|
tokens[i].setPriority(rand()/((double)RAND_MAX));
|
||
|
|
controller.addToken(&tokens[i]);
|
||
|
|
}
|
||
|
|
|
||
|
|
|
||
|
|
controller.start();
|
||
|
|
|
||
|
|
//change priorities
|
||
|
|
for(int i = 0; i < tokens.size(); i++)
|
||
|
|
tokens[i].setPriority(rand());
|
||
|
|
controller.updatePriorities();
|
||
|
|
|
||
|
|
//lock and use the tokens
|
||
|
|
for(int i = 0; i < tokens.size(); i++) {
|
||
|
|
bool success = tokens[i].lock();
|
||
|
|
if(success) {
|
||
|
|
//use the token as you see fit
|
||
|
|
tokens[i].unlock();
|
||
|
|
}
|
||
|
|
}
|
||
|
|
</pre>
|
||
|
|
|
||
|
|
|
||
|
|
</div>
|
||
|
|
</body>
|
||
|
|
</html>
|