snipt

Ctrl+h for KB shortcuts

Bash

Crazy rsync file level de-duplication

1
2
3
4
5
6
7
8
9
# from http://ask.slashdot.org/comments.pl?sid=2604202&cid=38589562

# First I sync from the remote directory to a local base directory:

rsync --partial -z -vlhprtogH --delete [email protected]:/etc/ /backup/server/www/etc/base/

#Then I sync that to the daily backup. Files that have not changed are hard-linked between all the days that share them. It very efficient and simple, and retrieving files is as simple as doing a directory search.

rsync -vlhprtogH --delete --link-dest=/backup/server/www/etc/base/ /backup/server/www/etc/base/ /backup/server/www/etc/2012-01-04
https://snipt.net/embed/2fae3f01cf5ccf9a420ee7fd516280e5/
/raw/2fae3f01cf5ccf9a420ee7fd516280e5/
2fae3f01cf5ccf9a420ee7fd516280e5
bash
Bash
10
2019-06-17T08:55:37
True
False
False
/api/public/snipt/34339/
crazy-rsync-file-level-de-duplication
<table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><a href="#L-1">1</a> <a href="#L-2">2</a> <a href="#L-3">3</a> <a href="#L-4">4</a> <a href="#L-5">5</a> <a href="#L-6">6</a> <a href="#L-7">7</a> <a href="#L-8">8</a> <a href="#L-9">9</a></pre></div></td><td class="code"><div class="highlight"><pre><span></span><span id="L-1"><a name="L-1"></a><span class="c1"># from http://ask.slashdot.org/comments.pl?sid=2604202&amp;cid=38589562</span> </span><span id="L-2"><a name="L-2"></a> </span><span id="L-3"><a name="L-3"></a><span class="c1"># First I sync from the remote directory to a local base directory:</span> </span><span id="L-4"><a name="L-4"></a> </span><span id="L-5"><a name="L-5"></a>rsync --partial -z -vlhprtogH --delete [email protected]:/etc/ /backup/server/www/etc/base/ </span><span id="L-6"><a name="L-6"></a> </span><span id="L-7"><a name="L-7"></a><span class="c1">#Then I sync that to the daily backup. Files that have not changed are hard-linked between all the days that share them. It very efficient and simple, and retrieving files is as simple as doing a directory search.</span> </span><span id="L-8"><a name="L-8"></a> </span><span id="L-9"><a name="L-9"></a>rsync -vlhprtogH --delete --link-dest<span class="o">=</span>/backup/server/www/etc/base/ /backup/server/www/etc/base/ /backup/server/www/etc/2012-01-04 </span></pre></div> </td></tr></table>
backup, rsync