pisces.schema.util

Helper functions for maintaining database schema.

class pisces.schema.util.CoreTable(name, prototype, table)

Bases: tuple

Attributes:

name

Alias for field number 0

prototype

Alias for field number 1

table

Alias for field number 2

Methods

count(value, /)	Return number of occurrences of value.
index(value[, start, stop])	Return first index of value.

name

Alias for field number 0

prototype

Alias for field number 1

table

Alias for field number 2

class pisces.schema.util.PiscesMeta(clsname, parents, dct)

Bases: DeclarativeMeta

Methods

__call__(*args, **kwargs)	Call self as a function.
mro(/)	Return a type's method resolution order.

pisces.schema.util.from_string(cls, line, default_on_error=None)

Construct a mapped table instance from correctly formatted flat file line.

Works with fixed-length fields, separated by a single whitespace.

Parameters:

line: str

Flat file line (best to remove newline, but maybe not a problem).

default_on_error: list, optional

Supply a list of column names that return default values if they produce an error during parsing (e.g. lddate).

Raises:

ValueError: Can’t properly parse the line.

Notes

default_on_error is useful for malformed fields, but it will also mask other problems with line parsing. It’s better to pre-process tables to match the table specifications or catch exceptions and isolate these lines.

Examples

>>> with open('TA.site','r') as ffsite:
        for line in ffsite:
            isite = Site.from_string(line)
or
>>> with open('TA.site','r') as ffsite:
        for line in ffsite:
            isite = Site.from_string(line, default_on_error=['lddate'])

pisces.schema.util.get_infovals(meta, structure, key)

Get flattened values from column info dictionary for given metadata and desired structure.

If key is not in column info, None is used.

Returns:

colnames: list

Flattened column names.

vals: list

Examples

>>> from pisces.schema.kbcore import Base
>>> fields, defaults = get_infovals(Base.metadata, ['site'], 'default')
>>> print(defaults)
['-', -1, 2286324, -999.0, -999.0, -999.0, '-', '-', '-', -99999, -99999, <built-in method now of type object at 0x100601060>]
>>> print(fields)
['sta', 'ondate', 'offdate', 'lat', 'lon', 'elev', 'staname', 'statype', 'refsta', 'dnorth', 'deast', 'lddate']

pisces.schema.util.parse_float(s)

pisces.schema.util.parse_int(s)

pisces.schema.util.parse_str(s)

pisces.schema.util.parse_utf(s)

Decode a utf-8 encoded string.

pisces.schema.util.string_formatter(meta, structure)

Get string substitution formatter for given structure and schema.

If you’re looking for a write_flatfile function, this is it. It takes advantage of native python string formatting and file writing.

Parameters:

meta: sqlalchemy.MetaData

Columns in tables must have info={‘format’: ?}

structure: iterable of str

Strings are known schema objects (tables, columns).

Returns:

fmt: str

String substitution formatter for provided structure within provided schema.

Examples

>>> recs = session.query(Origin, sta.lat, sta.lon).limit(10).all()
>>> fmt = string_formatter(meta.tables.values(), ['origin', 'lat', 'lon'])
>>> with open('origin_lat_lon.txt', 'w') as f:
        for rec in recs:
            f.write(fmt.format(rec))