Python : compress shelve - 圧縮機能付永続辞書
2007.01.20
永続辞書を実現するshelveモジュールに圧縮機能を付けたgzshelveというのを作った.
以下にソースコードを示す.shelve.open() の代わりにgzsheleve.open() を使えば圧縮機能付き永続辞書になる.ただし,速度は期待しないでください….
# -*- coding: utf-8 -*- import shelve, anydbm from zlib import compress, decompress try: from cPickle import Pickler, Unpickler except ImportError: from pickle import Pickler, Unpickler try: from cStringIO import StringIO except ImportError: from StringIO import StringIO class gzShelf(shelve.Shelf) : def __init__(self, dict, protocol=None, writeback=False, binary=None) : shelve.Shelf.__init__(self, dict, protocol, writeback, binary) def __getitem__(self, key) : try: value = self.cache[key] except KeyError: f = StringIO(decompress(self.dict[key])) value = Unpickler(f).load() if self.writeback: self.cache[key] = value return value def __setitem__(self, key, value): if self.writeback: self.cache[key] = value f = StringIO() p = Pickler(f, self._protocol) p.dump(value) self.dict[key] = compress(f.getvalue()) def open(filename, flag='c', protocol=None, writeback=False, binary=None) : db = anydbm.open(filename, flag) return gzShelf(db, protocol, writeback, binary)
カテゴリー:技術情報メモ