[hojia] Useful cronjobs for Zope-databases

[Carsten Raddatz]

Hello all,

I enjoyed last week's meeting; I hope you did as well. Whenever somebody I met is around Berlin let me know, there would be plenty of opportunities to meet open source people here if some time for planning is given.

With Ping I had a brief discussion about Zope and related maintenance questions. For me, the easiest way to handle the .fs files is to set up cronjobs that do the following, using only tools supplied with zope, i.e. repozo.py in /your/zope-install/path/Zope-2.9.4/bin/repozo.py (owned by the user the zope runs as) and the python you have anyway. We'd do this:
1. backup the ZODB incrementally, daily,
2. backup the ZODB in full (and zipped with -z), weekly,
3. "pack" the ZODB when needed, how often depens on site usage, typical workflow cycles etc. Replace foo with 30 days or whatever.
The paths here a customized, as the developers' recommendation is to compile the whole bunch by hand, including python. You usually end up in /opt/Zope-Version/ or else a user's home. Providing full paths may be anal, but lowers implicit requirements.
http://wiki.zope.org/zope2/RepozoPy and http://wiki.zope.org/ZODB/FileStorageBackup ("Backing up FileStorage files") provide a good read. They also explain how to use repozo.py to use the backed up files (add -o /path/to/new/Data.fs for that).
It should be noted that if the .fs file did not change, it does not back up incrementally, i.e. no deltafs is generated. Depending on the setup you may have to export PYTONPATH and explicitly unset PYTONPATH after doing the cronjob.

The cronjobs:

1. daily backup the ZODB incrementally
50  3  *   *   1-6     /srv/zope-install/python/Python-2.4.3/bin/python /srv/zope-backup/Zope-2.9.4/bin/repozo.py -B -v -r /srv/zope-backup/Zope-2.9.4/backups/domain.tld/ -f /srv/zope-instanz/var/Data-domain-tld.fs >> /srv/zope-instanz/log/repozo-backup-domain-tld-lokal.log

2.
50  3   *   *   0       /srv/zope-install/python/Python-2.4.3/bin/python /srv/zope-backup/Zope-2.9.4/bin/repozo.py -B -F -z -v -r /srv/zope-backup/Zope-2.9.4/backups/domain.de/ -f /srv/zope-instanz/var/Data-domain-de.fs >> /srv/zope-instanz/log/repozo-backup-domain-tld-lokal.log

3.
50 4 1 * *              ~/regulary-pack-all-zodb-files_main.py foo

where the regulary-pack-all-zodb-files_main.py reads:

#!/usr/bin/python -O

import sys, urllib

username="myzopeusername"
password="mypassword"
zope="http://localhost:port/"

class NoGUI_URLopener(urllib.FancyURLopener):
  def __init__(self, username, password, *args):
     apply(urllib.FancyURLopener.__init__, (self,) + args)

     self.username = username
     self.password = password
     self.asked = 0

  def prompt_user_passwd(self, host, realm):
     if self.asked:
        raise "Unauthorised"
     else:
        self.asked = 1
        return self.username, self.password

try:
  days = sys.argv[1]
except:
  days = 0

urllib._urlopener = NoGUI_URLopener(username, password)
urllib.urlretrieve("%s/Control_Panel/Database/manage_pack?days:float=%s" % (zope, days))

Hope that helps.

Carsten

--
Carsten Raddatz     cra at newthinking.de
http://www.newthinking-communications.de/
http://www.netzpolitik.org/
Tucholskystr. 48    fon: +49-30-288736-06
D-10117 Berlin      fax: +49-30-288736-07
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://tossug.org/pipermail/hojia/attachments/20070318/27b65486/attachment.bin