pisces.util#

Common Pisces utility functions.

pisces.util.add_rows(session, rows, recurse=False)[source]#

Handle common errors with logging in SQLAlchemy add_all.

Tries to add in bulk. Failing that, it will rollback and optionally try to add one at a time.

Parameters:

sessionsqlalchemy.orm.Session
rowslist: Mapped table instances.
recursebool, optional: After failure, try to add records individually.

Returns:

numint: Number of objects added. 0 if none.
eexception or None

pisces.util.db_connect(*args, **kwargs)[source]#

Connect to your database.

Parameters:

backendstring: One of the SQLAlchemy connection strings from http://docs.sqlalchemy.org/en/rel_0_7/core/engines.html#database-urls
userstring, optional: Not required for sqlite.
passwdstring, optional: Not needed for sqlite. Prompted if needed and not provided.
serverstring, optional: Database host server.
portstring or integer, optional: Port on remote server.
instancestring, optional: The database instance. For sqlite, this is the file name.
connstring, optional: A fully-formed SQLAlchemy style connection string.

Returns:

sessionbound SQLAlchemy Session instance

Notes

For connection string format, see: http://docs.sqlalchemy.org/en/rel_0_8/core/engines.html

Examples

Connect to a local sqlite database file:

>>> meta, session = db_connect(conn='sqlite:///mydb.sqlite')
#or
>>> meta, session = db_connect(backend='sqlite', instance='mydb.sqlite')

pisces.util.deprecated(instructions)[source]#

Flags a method as deprecated.

Parameters:

instructionsstr: A human-friendly string of instructions, such as: ‘Please migrate to add_proxy() ASAP.’

References

https://gist.github.com/kgriffs/8202106

pisces.util.gen_id(i=0)[source]#

Produce a generator for sequential integer id values.

Examples

>>> lastorid = 7
>>> orid = gen_id(lastorid)
>>> orid.next()
8
>>> orid.next()
9

Generate more than one at a time:

>>> orid, arid, wfid = (gen_id() for id in ['orid', 'arid', 'wfid'])
>>> orid.next(), arid.next()
(1, 1)

Dictionary of id generators for desired ids, starting where they left off. ids not in Lastid will be missing

>>> ids = session.query(Lastid).filter(Lastid.keyname.in_(['orid','arid']).all()
>>> last = dict([(id.keyname, gen_id(id.keyvalue)) for id in ids])
>>> last['orid'].next()
8820005

pisces.util.get_lastids(session, Lastid, keynames=None, expunge=True, create=False)[source]#

Load or create Lastid instances into a convenient and readable attribute-based dictionary.

Parameters:

sessionsqlalchemy.orm.Session instance
LastidLastid table class
idslist or tuple of strings: Lastid.keyname values to load.
expungebool: If True, expunge loaded ids from the session. This frees you to modify them without affecting the database from which they were loaded. In this case, you’ll have to add them back into a session and commit them for their changes to be reflected on the database.
createbool: If True, create ids that don’t already exist.

Returns:

lastobspy.core.AttribDict: Attribute-based dictionary of all lastids.

Examples

Get and set lastid values directly by name or by attribute. >>> last = get_lastids(session, Lastid, [‘orid’, ‘arid’]) >>> last.orid, last[‘orid’] Lastid(keyname=’orid’), Lastid(keyname=’orid’)

Test for their existence by name. >>> ‘orid’ in last True

Use the Lastid’s ‘next’ generator behavior for readable code >>> next(last.orid) 18 >>> last.orid.keyvalue 18

Update your database when you’re done. >>> session.add_all(ids.values()) >>> session.commit()

pisces.util.get_options(db, prefix=None)[source]#

for coretable in CORETABLES:

table_group.add_argument(’–’ + coretable.name,

default=None,: metavar=’owner.tablename’, dest=coretable.name)

pisces.util.get_or_create_tables(session, create=True, **tables)[source]#

Load or create canonical ORM KB Core table classes.

Parameters:

sessionsqlalchemy.orm.Session
createbool: If True, create a table object that isn’t found.
tables: Canonical table name / formatted table name keyword pairs.
Also accepted are canonical table name keywords with ‘[owner.]tablename’
arguments, which will replace any prefix-based core table names.

Returns:

tablesdict: Mapping between canonical table names and SQLA ORM classes. e.g. {‘origin’: MyOrigin, …}

pisces.util.get_tables(bind, fulltablenames, metadata=None, primary_keys=None, base=None)[source]#

Reflect/load an arbitrary database table as a mapped class.

This is a shortcut for SQLAlchemy’s declarative mapping using __table__. See http://docs.sqlalchemy.org/en/rel_0_9/orm/extensions/declarative.html#using-a-hybrid-approach-with-table.

Parameters:

bindsqlalchemy.engine.base.Engine instance: Engine pointing to the target database.
fulltableslist of strings: Of the form [‘owner1.tablename1’, ‘owner2.tablename2’, …] Leave out ‘owner.’ if database doesn’t use owners (sqlite, etc…)
metadatasqlalchemy.MetaData, optional: MetaData into which reflected Tables go. If not supplied, a new one is created, accessible from MyTable.metadata on one of the loaded tables.
primary_keysdict, optional: Tablename, primary key list pairs of the form, {‘owner1.tablename1’: [‘primary_key1’, ‘primary_key2’]} These are required if the table is a view or has no primary key.
basesqlalchemy.ext.declarative.api.DeclarativeMeta, optional: The declarative base the from which loaded table classes will inherit. The info dictionary of loaded Columns will be updated from those in the base. These are used to generate default values and string representations. Import from pisces.schema.css3, or extensions thereof. Default, sqlalchemy.ext.declarative.api.DeclarativeMeta.

Returns:

list: Corresponding list of ORM table classes mapped to reflected tables, Can be used for querying or making row instances.

Raises:

sqlalchemy.exc.NoSuchTableErrorTable doesn’t exist.
sqlalchemy.exc.InvalidRequestErrorTable already loaded in metadata.
sqlalchemy.exc.ArgumentErrorTable has no primary key(s).

Notes

In SQLAlchemy, a database account/owner is generally used with the “schema” keyword argument.

For core tables in a Pisces schema, this function isn’t recommended. Instead, subclass from the known abstract table.

Examples

# for unknown table >>> import pisces.schema.css3 as css >>> RandomTable = get_tables(engine, [‘randomtable’])

# for a known/prototype table >>> class Site(css.Site):

__tablename__ = ‘myaccount.my_site_tablename’

pisces.util.load_config(config)[source]#

Take dictionary-like object (actual dictionary or ConfigParser section) to produce a SQLAlchemy database session and dictionary of database table classes.

Parameters:

configdict-like

Returns:

sessionSQLAlchemy database session instance
tablesdict: Keys are canonical table names, values are SQLAlchemy table classes.
The config input is a dict-like object of table name keys and class paths.
This can be a configparser object or a nested dict. The “database” section and “url” key
are required. Any additional parameters in the section must be importable
module names, with optional SQLAlchemy table class names separated by a colon “:”.
If no class name is provided, a class with the capitalized key name is imported.

Examples

The following inputs should be equivalent.

# pisces.cfg [database] # required url = oracle://myuser:mypass@dbserver.someplace.gom:1521/dbname # required site = mytablemodule:SiteClassName wfdisc = mytablemodule:WfdiscClassName mytable = custom_tables:MyCustomTableClass arrival = othermodule

>>> config = {'url': 'oracle://myuser:mypass@dbserver.someplace.gom:1521/dbname',
              'site': 'modulename:SiteClassName',
              'wfdisc': 'modulename:Wfdisc_raw',
              'mytable': 'custom_tables:MyCustomTableClass',
              'arrival': 'othermodule',
              }
# or
>>> import configparser
>>> config = configparser.ConfigParser()
>>> config.read('pisces.cfg')
>>> session, tables = load_config(config['database'])
>>> tables
... {'arrival': othermodule.Arrival,
... 'mytable': custom_tables.MyCustomTableClass,
... 'site': modulename.SiteClassName,
... 'wfdisc': modulename.Wfdisc_raw}

pisces.util.make_same_size(lat1, lon1, lat2, lon2)[source]#: Returns numpy arrays the same size as longest inputs. assume: lat1/lon1 are same size and lat2/lon2 are same size assume: the smaller of the sizes is a scalar

pisces.util.make_table(fulltablename, prototype)[source]#

Create a new ORM class/model on-the-fly from a prototype.

Parameters:

fulltablenamestr: Schema-qualified name of the database table, like ‘owner.tablename’ or just ‘tablename’. The resulting classname will be the capitalized tablename, like ‘Tablename’.
prototypesqlalchemy abstract ORM class: The prototype table class. pisces.schema.css.Site, for example.

Notes

It’s better to declare classes in an external module, and import them. SQLAlchemy doesn’t let you use the same table names twice, so on-the-fly class creation and naming is risky:

You can’t use make_tables again if you accidentally overwrite the variable you used to hold the class you created.
You can’t use make_tables again if you import something from a script/module where make_tables was used with the same table name.

pisces.util.make_wildcard_list(toList)[source]#

Take a list, tuple, or comma separated string of variables and output a list with sql wildcards ‘%’ and ‘_’

Parameters:

toListlist, tuple, or comma separated string or variables

Returns:

nowList: list of variables contained in the toList variable

pisces.util.travel_times(ref, deg=None, km=None, depth=0.0)[source]#

Get approximate relative travel time(s).

Parameters:

reflist or tuple of strings and/or floats: Reference phase names or horizontal velocities [km/sec].
degfloat, optional: Degrees of arc between two points of interest (spherical earth).
kmfloat, optional: Horizontal kilometers between two points of interest (spherical earth).
depthfloat, optional. default, 0.: Depth (positive down) of event, in kilometers.

Returns:

numpy.ndarray: Relative times, in seconds, same length as “ref”. NaN if requested time is undefined.

Notes

Either deg or km must be indicated. The user is responsible for adding/subtracting time (such as origin time, pre-window noise time, etc.) from those predicted in order to define a window. Phase travel times use ak135.

Examples

Get relative P arrival and 2.7 km/sec surface wave arrival at 35 degrees distance. >>> times = travel_times([‘P’, 2.7], deg=35.0) To get absolute window, add the origin time like: >>> w1, w2 = times + epoch_origin_time

pisces.util.url_connect(url)[source]#

Connect to a database using an RFC-1738 compliant URL, like sqlalchemy’s create_engine, prompting for a password if a username is supplied.

Parameters:

urlstring: A fully-formed SQLAlchemy style connection string. See http://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls

Returns:

sessionbound SQLAlchemy Session instance

Examples

SQLite database file, local: >>> url_connect(‘sqlite:///local/path/to/mydb.sqlite’)

SQLite database file, full path: >>> url_connect(‘sqlite:////full/path/to/mydb.sqlite’)

Remote Oracle, OS-authenticated (no user or password needs to be specified) >>> url_connect(‘oracle://dbserver.lanl.gov:8080/mydb’)

Remote Oracle, password-authenticated (specify user, prompted for password) >>> url_connect(‘oracle://scott@dbserver.lanl.gov:8080/mydb’) Enter password for scott:

Remote Oracle, password-authenticated (password specified) >>> url_connect(‘oracle://scott:tiger@dbserver.lanl.gov:8080/mydb’)

pisces.util

Contents

pisces.util#