python3-memoizedb

2016-01-28 20:21 UTC
  • Xyne

Metadata

Description:

Generic data retrieval memoizer that uses an sqlite database to cache data.

Latest Version:

2015.12

Architecture:

  • any

Dependencies:

  • python3

Arch Repositories:

  • [xyne-any]
  • [xyne-i686]
  • [xyne-x86_64]

AUR Page:

python3-memoizedb

Tags:

About

MemoizeDB provides a way to cache results that should not be regenerated or reretrieved every time they are accessed. It was originally written to cache JSON data retrieved from a web interface to avoid placing unnecessary load on the server for results that were unlikely to change between each query.

You can determine how long results should be saved in the database before they are refreshed.

Example

The following script will retreive package data from the AUR. You will notice that it is slower the first time it displays information for a given package name but subsequent queries within the set caching period are much faster.

#!/usr/bin/env python3

import json
import MemoizeDB
import sqlite3
import sys
import urllib.parse
import urllib.request


# The retriever function.
def get_pkginfo(name):
  encoded_name = urllib.parse.quote(name)
  url = 'https://aur.archlinux.org/rpc.php?type=info&arg={}'.format(encoded_name)
  with urllib.request.urlopen(url) as f:
    return (f.read().decode(),)

# The glue for handling the database. 300: cache the data for 5 minutes.
glue = {
  'pkginfo' : (get_pkginfo, (('json', 'TEXT'),), 300)
}

# The connection.
conn = sqlite3.connect(
  'test.sqlite3',
  detect_types=(sqlite3.PARSE_DECLTYPES | sqlite3.PARSE_COLNAMES)
)
mdb = MemoizeDB.MemoizeDB(conn, glue)
mdb.db_initialize()

# A wrapper function to convert the JSON text to an object.
def get_obj(name):
  txt = mdb.get_one('pkginfo', name)
  return json.loads(txt)

if sys.argv[1:]:
  names = sys.argv[1:]
else:
  names = ('python3-memoizedb', 'python3-aur', 'powerpill', 'bauerbill')

# Display the data.
for name in names:
  obj = get_obj(name)
  print(json.dumps(obj, indent=2, sort_keys=True))

CHANGELOG

2012-11-03

  • fixed documentation for glue function in MemoizeDB's help text
  • added null_ttl parameter for temporarily caching empty data
  • removed support for non-tuple values to db_insert method
Contact
echo xyne.archlinux.ca | sed 's/\./@/'
Feeds
Blog News
Validation
XHTML 1.0 Strict CSS level 3 Atom 1.0