Source code for xdress.typesystem

"""Implements a simple, dynamic type system for API generation.

:author: Anthony Scopatz <scopatz@gmail.com>

Introduction
============

This module provides a suite of tools for denoting, describing, and converting
between various data types and the types coming from various systems.  This is
achieved by providing canonical abstractions of various kinds of types:

* Base types (int, str, float, non-templated classes)
* Refined types (even or odd ints, strings containing the letter 'a')
* Dependent types (templates such arrays, maps, sets, vectors)

All types are known by their name (a string identifier) and may be aliased with
other names.  However, the string id of a type is not sufficient to fully describe
most types.  The system here implements a canonical form for all kinds of types.
This canonical form is itself hashable, being comprised only of strings, ints,
and tuples.

Canonical Forms
---------------
First, let us examine the base types and the forms that they may take.  Base types
are fiducial.  The type system itself may not make any changes (refinements,
template filling) to types of this kind.  They are basically a collection of bits.
(The job of ascribing meaning to these bits falls on someone else.)  Thus base types
may be referred to simply by their string identifier.  For example::

    'str'
    'int32'
    'float64'
    'MyClass'

Aliases to these -- or any -- type names are given in the type_aliases dictionary::

    type_aliases = {
        'i': 'int32',
        'i4': 'int32',
        'int': 'int32',
        'complex': 'complex128',
        'b': 'bool',
        }

Furthermore, length-2 tuples are used to denote a type and the name or flag of its
predicate.  A predicate is a function or transformation that may be applied to
verify, validate, cast, coerce, or extend a variable of the given type.  A common
usage is to declare a pointer or reference of the underlying type.  This is done with
the string flags '*' and '&'::

    ('char', '*')
    ('float64', '&')

If the predicate is a positive integer, then this is interpreted as a
homogeneous array of the underlying type with the given length.  If this length
is zero, then the tuple is often interpreted as a scalar of this type, equivalent
to the type itself.  The length-0 scalar interpretation depends on context.  Here
are some examples of array types::

    ('char', 42)  # length-42 character array
    ('bool', 1)   # length-1 boolean array
    ('f8', 0)     # scalar 64-bit float

.. note::

    length-1 tuples are converted to length-2 tuples with a 0 predicate,
    i.e. ``('char',)`` will become ``('char', 0)``.

The next kind of type are **refinement types** or **refined types**.  A refined type
is a sub-type of another type but restricts in some way what constitutes a valid
element.  For example, if we first take all integers, the set of all positive
integers is a refinement of the original.  Similarly, starting with all possible
strings the set of all strings starting with 'A' is a refinement.

In the system here, refined types are given their own unique names (e.g. 'posint'
and 'astr').  The type system has a mapping (``refined_types``) from all refinement
type names to the names of their super-type.  The user may refer to refinement types
simply by their string name.  However the canonical form refinement types is to use
the refinement as the predicate of the super-type in a length-2 tuple, as above::

    ('int32', 'posint')  # refinement of integers to positive ints
    ('str', 'astr')      # refinement of strings to str starting with 'A'

It is these refinement types that give the second index in the tuple its 'predicate'
name.  Additionally, the predicate is used to look up the converter and validation
functions when doing code generation or type verification.

The last kind of types are known as **dependent types** or **template types**,
similar in concept to C++ template classes.  These are meta-types whose
instantiation requires one or more parameters to be filled in by further values or
types. Dependent types may nest with themselves or other dependent types.  Fully
qualifying a template type requires the resolution of all dependencies.

Classic examples of dependent types include the C++ template classes.  These take
other types as their dependencies.  Other cases may require only values as
their dependencies.  For example, suppose we want to restrict integers to various
ranges.  Rather than creating a refinement type for every combination of integer
bounds, we can use a single 'intrange' type that defines high and low dependencies.

The ``template_types`` mapping takes the dependent type names (e.g. 'map')
to a tuple of their dependency names ('key', 'value').   The ``refined_types``
mapping also accepts keys that are tuples of the following form::

    ('<type name>', '<dep0-name>', ('dep1-name', 'dep1-type'), ...)

Note that template names may be reused as types of other template parameters::

    ('name', 'dep0-name', ('dep1-name', 'dep0-name'))

As we have seen, dependent
types may either be base types (when based off of template classes), refined types,
or both.  Their canonical form thus follows the rules above with some additional
syntax.  The first element of the tuple is still the type name and the last
element is still the predicate (default 0).  However the type tuples now have a
length equal to 2 plus the number of dependencies.  These dependencies are
placed between the name and the predicate: ``('<name>', <dep0>, ..., <predicate>)``.
These dependencies, of course, may be other type names or tuples!  Let's see
some examples.

In the simplest case, take analogies to C++ template classes::

    ('set', 'complex128', 0)
    ('map', 'int32', 'float64', 0)
    ('map', ('int32', 'posint'), 'float64', 0)
    ('map', ('int32', 'posint'), ('set', 'complex128', 0), 0)

Now consider the intrange type from above.  This has the following definition and
canonical form::

    refined_types = {('intrange', ('low', 'int32'), ('high', 'int32')): 'int32'}

    # range from 1 -> 2
    ('int32', ('intrange', ('low', 'int32', 1), ('high', 'int32', 2)))

    # range from -42 -> 42
    ('int32', ('intrange', ('low', 'int32', -42), ('high', 'int32', 42)))

Note that the low and high dependencies here are length three tuples of the form
``('<dep-name>', <dep-type>, <dep-value>)``.  How the dependency values end up
being used is solely at the discretion of the implementation.  These values may
be anything, though they are most useful when they are easily convertible into
strings in the target language.

.. warning::

    Do not confuse length-3 dependency tuples with length-3 type tuples!
    The last element here is a value, not a predicate.

Next, consider a 'range' type which behaves similarly to 'intrange' except that
it also accepts the type as dependency.  This has the following definition and
canonical form::

    refined_types = {('range', 'vtype', ('low', 'vtype'), ('high', 'vtype')): 'vtype'}

    # integer range from 1 -> 2
    ('int32', ('range', 'int32', ('low', 'int32', 1), ('high', 'int32', 2)))

    # positive integer range from 42 -> 65
    (('int32', 'posint'), ('range', ('int32', 'posint'),
                                    ('low', ('int32', 'posint'), 42),
                                    ('high', ('int32', 'posint'), 65)))

Shorthand Forms
---------------
The canonical forms for types contain all the information needed to fully describe
different kinds of types.  However, as human-facing code, they can be exceedingly
verbose.  Therefore there are number of shorthand techniques that may be used to
also denote the various types.  Converting from these shorthands to the fully
expanded version may be done via the the ``canon(t)`` function.  This function
takes a single type and returns the canonical form of that type.  The following
are operations that ``canon()``  performs:

* Base type are returned as their name::

    canon('str') == 'str'

* Aliases are resolved::

    canon('f4') == 'float32'

* Expands length-1 tuples to scalar predicates::

    t = ('int32',)
    canon(t) == ('int32', 0)

* Determines the super-type of refinements::

    canon('posint') == ('int32', 'posint')

* Applies templates::

    t = ('set', 'float')
    canon(t) == ('set', 'float64', 0)

* Applies dependencies::

    t = ('intrange', 1, 2)
    canon(t) = ('int32', ('intrange', ('low', 'int32', 1), ('high', 'int32', 2)))

    t = ('range', 'int32', 1, 2)
    canon(t) = ('int32', ('range', 'int32', ('low', 'int32', 1), ('high', 'int32', 2)))

* Performs all of the above recursively::

    t = (('map', 'posint', ('set', ('intrange', 1, 2))),)
    canon(t) == (('map',
                 ('int32', 'posint'),
                 ('set', ('int32',
                    ('intrange', ('low', 'int32', 1), ('high', 'int32', 2))), 0)), 0)

These shorthands are thus far more useful and intuitive than canonical form described
above.  It is therefore recommended that users and developers write code that uses
the shorter versions, Note that ``canon()`` is guaranteed to return strings, tuples,
and integers only -- making the output of this function hashable.

Built-in Template Types
-----------------------
Template type definitions that come stock with xdress::

    template_types = {
        'map': ('key_type', 'value_type'),
        'dict': ('key_type', 'value_type'),
        'pair': ('key_type', 'value_type'),
        'set': ('value_type',),
        'list': ('value_type',),
        'tuple': ('value_type',),
        'vector': ('value_type',),
        }

Built-in Refined Types
-----------------------
Refined type definitions that come stock with xdress::

    refined_types = {
        'nucid': 'int32',
        'nucname': 'str',
        ('enum', ('name', 'str'), ('aliases', ('dict', 'str', 'int32', 0))): 'int32',
        ('function', ('arguments', ('list', ('pair', 'str', 'type'))), ('returns', 'type')): 'void',
        ('function_pointer', ('arguments', ('list', ('pair', 'str', 'type'))), ('returns', 'type')): ('void', '*'),
        }

Major Classes Overview
----------------------
Holistically, the following classes are important to type system:

* ``TypeSystem``:  This *is* the type system.
* ``TypeMatcher``: An imutable type for matching types against a pattern.
* ``MatchAny``: A singleton used to denote patterns.
* ``typestr``: Various string representations of a type as properties.

Type System API
===============

"""
from __future__ import print_function
import os
import io
import sys
from contextlib import contextmanager
from collections import Sequence, Set, Iterable, MutableMapping, Mapping
from numbers import Number
from pprint import pprint, pformat
from warnings import warn
import gzip
try:
    import cPickle as pickle
except ImportError:
    import pickle

from .utils import Arg, flatten, indent, memoize_method, infer_format

if sys.version_info[0] >= 3:
    basestring = str

[docs]class TypeSystem(object): """A class representing a type system. """ datafields = set(['base_types', 'template_types', 'refined_types', 'humannames', 'extra_types', 'dtypes', 'stlcontainers', 'argument_kinds', 'variable_namespace', 'type_aliases', 'cpp_types', 'numpy_types', 'from_pytypes', 'cython_ctypes', 'cython_cytypes', 'cython_pytypes', 'cython_cimports', 'cython_cyimports', 'cython_pyimports', 'cython_functionnames', 'cython_classnames', 'cython_c2py_conv', 'cython_py2c_conv']) def __init__(self, base_types=None, template_types=None, refined_types=None, humannames=None, extra_types='xdress_extra_types', dtypes='dtypes', stlcontainers='stlcontainers', argument_kinds=None, variable_namespace=None, type_aliases=None, cpp_types=None, numpy_types=None, from_pytypes=None, cython_ctypes=None, cython_cytypes=None, cython_pytypes=None, cython_cimports=None, cython_cyimports=None, cython_pyimports=None, cython_functionnames=None, cython_classnames=None, cython_c2py_conv=None, cython_py2c_conv=None, typestring=None): """Parameters ---------- base_types : set of str, optional The base or primitive types in the type system. template_types : dict, optional Template types are types whose instantiations are based on meta-types. this dict maps their names to meta-type names in order. refined_types : dict, optional This is a mapping from refinement type names to the parent types. The parent types may either be base types, compound types, template types, or other refined types! humannames : dict, optional The human readable names for types. extra_types : str, optional The name of the xdress extra types module. dtypes : str, optional The name of the xdress numpy dtypes wrapper module. stlcontainers : str, optional The name of the xdress C++ standard library containers wrapper module. argument_kinds : dict, optional Templates types have arguments. This is a mapping from type name to a tuple of utils.Arg kind flags. The order in the tuple matches the value in template_types. This is only vaid for concrete types, ie ('vector', 'int', 0) and not just 'vector'. variable_namespace : dict, optional Templates arguments may be variables. These variables may live in a namespace which is required for specifiying the type. This is a dictionary mapping variable names to thier namespace. type_aliases : dict, optional Aliases that may be used to substitute one type name for another. cpp_types : dict, optional The C/C++ representation of the types. numpy_types : dict, optional NumPy's Cython representation of the types. from_pytypes : dict, optional List of Python types which match may be converted to these types. cython_ctypes : dict, optional Cython's C/C++ representation of the types. cython_cytypes : dict, optional Cython's Cython representation of the types. cython_pytypes : dict, optional Cython's Python representation of the types. cython_cimports : dict, optional A sequence of tuples representing cimports that are needed for Cython to represent C/C++ types. cython_cyimports : dict, optional A sequence of tuples representing cimports that are needed for Cython to represent Cython types. cython_pyimports : dict, optional A sequence of tuples representing imports that are needed for Cython to represent Python types. cython_functionnames : dict, optional Cython alternate name fragments used for mangling function and variable names. These should try to adhere to a lowercase_and_underscore convention. These may contain template argument namess as part of a format string, ie ``{'map': 'map_{key_type}_{value_type}'}``. cython_classnames : dict, optional Cython alternate name fragments used for mangling class names. These should try to adhere to a CapCase convention. These may contain template argument namess as part of a format string, ie ``{'map': 'Map{key_type}{value_type}'}``. cython_c2py_conv : dict, optional Cython convertors from C/C++ types to the representative Python types. cython_py2c_conv : dict, optional Cython convertors from Python types to the representative C/C++ types. Valuse are tuples with the form of ``(body or return, return or False)``. typestring : typestr or None, optional An type that is used to format types to strings in conversion routines. """ self.base_types = base_types if base_types is not None else set(['char', 'uchar', 'str', 'int16', 'int32', 'int64', 'int128', 'uint16', 'uint32', 'uint64', 'uint128', 'float32', 'float64', 'float128', 'complex128', 'void', 'bool', 'type', 'file', 'exception']) self.template_types = template_types if template_types is not None else { 'map': ('key_type', 'value_type'), 'dict': ('key_type', 'value_type'), 'pair': ('key_type', 'value_type'), 'set': ('value_type',), 'list': ('value_type',), 'tuple': ('value_type',), 'vector': ('value_type',), 'enum': ('name', 'aliases'), 'function': ('arguments', 'returns'), 'function_pointer': ('arguments', 'returns'), } self.refined_types = refined_types if refined_types is not None else { 'nucid': 'int32', 'nucname': 'str', ('enum', ('name', 'str'), ('aliases', ('dict', 'str', 'int32', 0))): 'int32', ('function', ('arguments', ('list', ('pair', 'str', 'type'))), ('returns', 'type')): 'void', ('function_pointer', ('arguments', ('list', ('pair', 'str', 'type'))), ('returns', 'type')): ('void', '*'), } self.humannames = humannames if humannames is not None else { 'char': 'character', 'uchar': 'unsigned character', 'str': 'string', 'bool': 'boolean', 'int16': 'short integer', 'int32': 'integer', 'int64': 'long integer', 'int128': 'very long integer', 'uint16': 'unsigned short integer', 'uint32': 'unsigned integer', 'uint64': 'unsigned long integer', 'uint128': 'unsigned very long integer', 'float32': 'float', 'float64': 'double', 'float128': 'long double', 'complex128': 'complex', 'file': 'file', 'exception': 'exception', 'dict': 'dict of ({key_type}, {value_type}) items', 'map': 'map of ({key_type}, {value_type}) items', 'pair': '({key_type}, {value_type}) pair', 'set': 'set of {value_type}', 'vector': 'vector [ndarray] of {value_type}', } self.extra_types = extra_types self.dtypes = dtypes self.stlcontainers = stlcontainers self.argument_kinds = argument_kinds if argument_kinds is not None else { ('vector', 'bool', 0): (Arg.TYPE,), ('vector', 'char', 0): (Arg.TYPE,), } self.variable_namespace = variable_namespace if \ variable_namespace is not None else {} self.type_aliases = _LazyConfigDict(type_aliases if type_aliases is not \ None else { 'i': 'int32', 'i2': 'int16', 'i4': 'int32', 'i8': 'int64', 'i16': 'int128', 'int': 'int32', 'ui': 'uint32', 'ui2': 'uint16', 'ui4': 'uint32', 'ui8': 'uint64', 'ui16': 'uint128', 'uint': 'uint32', 'f': 'float64', 'f4': 'float32', 'f8': 'float64', 'f16': 'float128', 'float': 'float64', 'double': 'float64', 'complex': 'complex128', 'b': 'bool', 'v': 'void', 's': 'str', 'string': 'str', 'FILE': 'file', '_IO_FILE': 'file', # 'c' has char / complex ambiguity, not included 'NPY_BYTE': 'char', 'NPY_UBYTE': 'uchar', 'NPY_STRING': 'str', 'NPY_INT16': 'int16', 'NPY_INT32': 'int32', 'NPY_INT64': 'int64', 'NPY_UINT16': 'uint16', 'NPY_UINT32': 'uint32', 'NPY_UINT64': 'uint64', 'NPY_FLOAT32': 'float32', 'NPY_FLOAT64': 'float64', 'NPY_COMPLEX128': 'complex128', 'NPY_BOOL': 'bool', 'NPY_VOID': 'void', 'NPY_OBJECT': 'void', 'np.NPY_BYTE': 'char', 'np.NPY_UBYTE': 'uchar', 'np.NPY_STRING': 'str', 'np.NPY_INT16': 'int16', 'np.NPY_INT32': 'int32', 'np.NPY_INT64': 'int64', 'np.NPY_UINT16': 'uint16', 'np.NPY_UINT32': 'uint32', 'np.NPY_UINT64': 'uint64', 'np.NPY_FLOAT32': 'float32', 'np.NPY_FLOAT64': 'float64', 'np.NPY_COMPLEX128': 'complex128', 'np.NPY_BOOL': 'bool', 'np.NPY_VOID': 'void', 'np.NPY_OBJECT': 'void', }, self) def cpp_types_function(t, ts): rtnct = ts.cpp_type(t[2][2]) argcts = [ts.cpp_type(argt) for n, argt in t[1][2]] if argcts == ['void']: argcts = [] return rtnct + " {type_name}(" + ", ".join(argcts) + ")" def cpp_types_function_pointer(t, ts): rtnct = ts.cpp_type(t[2][2]) argcts = [ts.cpp_type(argt) for n, argt in t[1][2]] if argcts == ['void']: argcts = [] return rtnct + " (*{type_name})(" + ", ".join(argcts) + ")" self.cpp_types = _LazyConfigDict(cpp_types if cpp_types is not None else { 'char': 'char', 'uchar': 'unsigned char', 'str': 'std::string', 'int16': 'short', 'int32': 'int', 'int64': 'long long', 'uint16': 'unsigned short', 'uint32': 'unsigned long', 'uint64': 'unsigned long long', 'float32': 'float', 'float64': 'double', 'float128': 'long double', 'complex128': '{extra_types}complex_t', 'bool': 'bool', 'void': 'void', 'file': 'FILE', 'exception': '{extra_types}exception', 'map': 'std::map', 'dict': 'std::map', 'pair': 'std::pair', 'set': 'std::set', 'vector': 'std::vector', True: 'true', 'true': 'true', 'True': 'true', False: 'false', 'false': 'false', 'False': 'false', 'function': cpp_types_function, 'function_pointer': cpp_types_function_pointer, }, self) self.numpy_types = _LazyConfigDict(numpy_types if numpy_types is not None \ else { 'char': 'np.NPY_BYTE', 'uchar': 'np.NPY_UBYTE', #'str': 'np.NPY_STRING', 'int16': 'np.NPY_INT16', 'int32': 'np.NPY_INT32', 'int64': 'np.NPY_INT64', 'uint16': 'np.NPY_UINT16', 'uint32': 'np.NPY_UINT32', 'uint64': 'np.NPY_UINT64', 'float32': 'np.NPY_FLOAT32', 'float64': 'np.NPY_FLOAT64', 'float128': 'np.NPY_FLOAT128', 'complex128': 'np.NPY_COMPLEX128', 'bool': 'np.NPY_BOOL', 'void': 'np.NPY_VOID', }, self) self.from_pytypes = from_pytypes if from_pytypes is not None else { 'str': ['basestring'], 'char': ['basestring'], 'uchar': ['basestring'], 'int16': ['int'], 'int32': ['int'], 'int64': ['int'], 'uint16': ['int'], 'uint32': ['int'], 'uint64': ['int'], 'float32': ['float', 'int'], 'float64': ['float', 'int'], 'complex128': ['complex', 'float'], 'file': ['file'], ('file', '*'): ['file'], 'set': ['collections.Set', 'list', 'basestring', 'tuple'], 'map': ['collections.Mapping', 'list', 'tuple'], 'vector': ['list', 'tuple', 'np.ndarray'], } def cython_ctypes_function(t, ts): rtnct = ts.cython_ctype(t[2][2]) argcts = [ts.cython_ctype(argt) for n, argt in t[1][2]] if argcts == ['void']: argcts = [] return rtnct + " {type_name}(" + ", ".join(argcts) + ")" def cython_ctypes_function_pointer(t, ts): rtnct = ts.cython_ctype(t[2][2]) argcts = [ts.cython_ctype(argt) for n, argt in t[1][2]] if argcts == ['void']: argcts = [] return rtnct + " (*{type_name})(" + ", ".join(argcts) + ")" self.cython_ctypes = _LazyConfigDict(cython_ctypes if cython_ctypes is not \ None else { 'char': 'char', 'uchar': '{extra_types}uchar', 'str': 'std_string', 'int16': 'short', 'int32': 'int', 'int64': '{extra_types}int64', 'uint16': '{extra_types}uint16', 'uint32': '{extra_types}uint32', 'uint64': '{extra_types}uint64', 'float32': 'float', 'float64': 'double', 'float128': '{extra_types}float128', 'complex128': '{extra_types}complex_t', 'bool': 'cpp_bool', 'void': 'void', 'file': 'c_file', 'exception': '{extra_types}exception', 'map': 'cpp_map', 'dict': 'dict', 'pair': 'cpp_pair', 'set': 'cpp_set', 'vector': 'cpp_vector', 'function': cython_ctypes_function, 'function_pointer': cython_ctypes_function_pointer, }, self) self.cython_cytypes = _LazyConfigDict(cython_cytypes if cython_cytypes \ is not None else { 'char': 'char', 'uchar': 'unsigned char', 'str': 'char *', 'int16': 'short', 'int32': 'int', ('int32', '*'): 'int *', 'int64': 'long long', 'uint16': 'unsigned short', 'uint32': 'unsigned long', # 'unsigned int' 'uint64': 'unsigned long long', 'float32': 'float', 'float64': 'float', 'float128': 'long double', 'complex128': 'object', 'bool': 'bool', 'void': 'void', 'file': 'c_file', 'exception': '{extra_types}exception', 'map': '{stlcontainers}_Map{key_type}{value_type}', 'dict': 'dict', 'pair': '{stlcontainers}_Pair{value_type}', 'set': '{stlcontainers}_Set{value_type}', 'vector': 'np.ndarray', 'function': 'object', 'function_pointer': 'object', }, self) self.cython_pytypes = _LazyConfigDict(cython_pytypes if cython_pytypes \ is not None else { 'char': 'str', 'uchar': 'str', 'str': 'str', 'int16': 'int', 'int32': 'int', 'int64': 'int', 'uint16': 'int', # 'unsigned int' 'uint32': 'int', # 'unsigned int' 'uint64': 'int', # 'unsigned int' 'float32': 'float', 'float64': 'float', 'float128': 'np.float128', 'complex128': 'object', 'file': 'file', 'exception': 'Exception', 'bool': 'bool', 'void': 'object', 'map': '{stlcontainers}Map{key_type}{value_type}', 'dict': 'dict', 'pair': '{stlcontainers}Pair{value_type}', 'set': '{stlcontainers}Set{value_type}', 'vector': 'np.ndarray', }, self) def cython_cimports_functionish(t, ts, seen): seen.add(('cython.operator', 'dereference', 'deref')) for n, argt in t[1][2]: ts.cython_cimport_tuples(argt, seen=seen, inc=('c',)) ts.cython_cimport_tuples(t[2][2], seen=seen, inc=('c',)) self.cython_cimports = _LazyImportDict(cython_cimports if cython_cimports \ is not None else { 'char': (None,), 'uchar': (('{extra_types}',),), 'str': (('libcpp.string', 'string', 'std_string'),), 'int16': (None,), 'int32': (None,), ('int32', '*'): 'int *', 'int64': (('{extra_types}',),), 'uint16': (('{extra_types}',),), 'uint32': (('{extra_types}',),), 'uint64': (('{extra_types}',),), 'float32': (None,), 'float64': (None,), 'float128': (('{extra_types}',),), 'complex128': (('{extra_types}',),), 'bool': (('libcpp', 'bool', 'cpp_bool'),), 'void': (None,), 'file': (('libc.stdio', 'FILE', 'c_file'),), 'exception': (('{extra_types}',),), 'map': (('libcpp.map', 'map', 'cpp_map'),), 'dict': (None,), 'pair': (('libcpp.utility', 'pair', 'cpp_pair'),), 'set': (('libcpp.set', 'set', 'cpp_set'),), 'vector': (('libcpp.vector', 'vector', 'cpp_vector'),), 'nucid': (('pyne', 'cpp_nucname'),), 'nucname': (('pyne', 'cpp_nucname'), ('libcpp.string', 'string', 'std_string')), 'function': cython_cimports_functionish, 'function_pointer': cython_cimports_functionish, }, self) def cython_cyimports_functionish(t, ts, seen): for n, argt in t[1][2]: ts.cython_cimport_tuples(argt, seen=seen, inc=('cy',)) ts.cython_cimport_tuples(t[2][2], seen=seen, inc=('cy',)) self.cython_cyimports = _LazyImportDict(cython_cyimports if \ cython_cyimports is not None else { 'char': (None,), 'uchar': (None,), 'str': (None,), 'int16': (None,), 'int32': (None,), 'int64': (None,), 'uint16': (None,), 'uint32': (None,), 'uint64': (None,), 'float32': (None,), 'float64': (None,), 'float128': (None,), 'complex128': (('{extra_types}',),), # for py2c_complex() 'bool': (None,), 'void': (None,), 'file': (('{extra_types}',),), 'exception': (('{extra_types}',),), 'map': (('{stlcontainers}',),), 'dict': (None,), 'pair': (('{stlcontainers}',),), 'set': (('{stlcontainers}',),), 'vector': (('numpy', 'as', 'np'), ('{dtypes}',)), 'nucid': (('pyne', 'nucname'),), 'nucname': (('pyne', 'nucname'),), 'function': cython_cyimports_functionish, 'function_pointer': cython_cyimports_functionish, }, self) def cython_pyimports_functionish(t, ts, seen): seen.add(('warnings',)) for n, argt in t[1][2]: ts.cython_import_tuples(argt, seen=seen) ts.cython_import_tuples(t[2][2], seen=seen) self.cython_pyimports = _LazyImportDict(cython_pyimports if \ cython_pyimports is not None else { 'char': (None,), 'uchar': (None,), 'str': (None,), 'int16': (None,), 'int32': (None,), 'int64': (None,), 'uint16': (None,), 'uint32': (None,), 'uint64': (None,), 'float32': (None,), 'float64': (None,), 'float128': (None,), 'complex128': (None,), 'bool': (None,), 'void': (None,), 'file': (None,), 'exception': (None,), 'map': (('{stlcontainers}',), ('collections',)), 'dict': (None,), 'pair': (('{stlcontainers}',),), 'set': (('{stlcontainers}',), ('collections',)), 'vector': (('numpy', 'as', 'np'),), 'nucid': (('pyne', 'nucname'),), 'nucname': (('pyne', 'nucname'),), 'function': cython_pyimports_functionish, 'function_pointer': cython_pyimports_functionish, }, self) self.cython_functionnames = _LazyConfigDict(cython_functionnames if \ cython_functionnames is not None else { # base types 'char': 'char', 'uchar': 'uchar', 'str': 'str', 'int16': 'short', 'int32': 'int', 'int64': 'long', 'uint16': 'ushort', 'uint32': 'uint', 'uint64': 'ulong', 'float32': 'float', 'float64': 'double', 'float128': 'longdouble', 'complex128': 'complex', 'bool': 'bool', 'void': 'void', 'file': 'file', 'exception': 'exception', # template types 'map': 'map_{key_type}_{value_type}', 'dict': 'dict', 'pair': 'pair_{key_type}_{value_type}', 'set': 'set_{value_type}', 'vector': 'vector_{value_type}', 'nucid': 'nucid', 'nucname': 'nucname', 'function': 'function', 'function_pointer': 'functionpointer', }, self) self.cython_classnames = _LazyConfigDict(cython_classnames if \ cython_classnames is not None else { # base types 'char': 'Char', 'uchar': 'UChar', 'str': 'Str', 'int32': 'Short', 'int32': 'Int', 'int64': 'Long', 'uint16': 'UShort', 'uint32': 'UInt', 'uint64': 'ULong', 'float32': 'Float', 'float64': 'Double', 'float128': 'Longdouble', 'complex128': 'Complex', 'bool': 'Bool', 'void': 'Void', 'file': 'File', 'exception': 'Exception', # template types 'map': 'Map{key_type}{value_type}', 'dict': 'Dict', 'pair': 'Pair{key_type}{value_type}', 'set': 'Set{value_type}', 'vector': 'Vector{value_type}', 'nucid': 'Nucid', 'nucname': 'Nucname', }, self) def cython_c2py_conv_function_pointer(t_, ts): """Wrap function pointers in C/C++ to Python functions.""" t = t_[1] argnames = [] argdecls = [] argbodys = [] argrtns = [] for n, argt in t[1][2]: argnames.append(n) decl, body, rtn = ts.cython_py2c(n, argt, proxy_name="c_" + n) argdecls += decl.split('\n') if isinstance(decl,basestring) else [decl] argbodys += body.split('\n') if isinstance(body,basestring) else [body] argrtns += rtn.split('\n') if isinstance(rtn,basestring) else [rtn] rtnname = 'rtn' rtnprox = 'c_' + rtnname rtncall = 'c_call_' + rtnname while rtnname in argnames or rtnprox in argnames: rtnname += '_' rtnprox += '_' argdecls = indent(argdecls) argbodys = indent(argbodys) rtndecl, rtnbody, rtnrtn, _ = ts.cython_c2py(rtncall, t[2][2], cached=False, proxy_name=rtnprox, existing_name=rtncall) if rtndecl is None and rtnbody is None: rtnprox = rtnname rtndecls = [rtndecl] returns_void = (t[2][2] == 'void') if not returns_void: rtndecls.append("cdef {0} {1}".format(ts.cython_ctype(t[2][2]), rtncall)) rtndecl = indent(rtndecls) rtnbody = indent(rtnbody) s = ('def {{proxy_name}}({arglist}):\n' '{argdecls}\n' '{rtndecl}\n' ' if {{var}} == NULL:\n' ' raise RuntimeError("{{var}} is NULL and may not be ' 'safely called!")\n' '{argbodys}\n') s += ' {{var}}({carglist})\n' if returns_void else \ ' {rtncall} = {{var}}({carglist})\n' s += '{rtnbody}\n' s = s.format(arglist=", ".join(argnames), argdecls=argdecls, cvartypeptr=ts.cython_ctype(t_).format(type_name='cvartype'), argbodys=argbodys, rtndecl=rtndecl, rtnprox=rtnprox, rtncall=rtncall, carglist=", ".join(argrtns), rtnbody=rtnbody) caches = 'if {cache_name} is None:\n' + indent(s) if not returns_void: caches += "\n return {rtnrtn}".format(rtnrtn=rtnrtn) caches += '\n {cache_name} = {proxy_name}\n' return s, s, caches self.cython_c2py_conv = _LazyConverterDict(cython_c2py_conv if \ cython_c2py_conv is not None else { # Has tuple form of (copy, [view, [cached_view]]) # base types 'char': ('chr(<int> {var})',), ('char', '*'): ('bytes({var}).decode()',), 'uchar': ('chr(<unsigned int> {var})',), ('uchar', '*'): ('bytes(<char *> {var}).decode()',), 'str': ('bytes(<char *> {var}.c_str()).decode()',), ('str', '*'): ('bytes(<char *> {var}[0].c_str()).decode()',), 'int16': ('int({var})',), ('int16', '*'): ('int({var}[0])',), 'int32': ('int({var})',), ('int32', '*'): ('int({var}[0])',), 'int64': ('int({var})',), ('int64', '*'): ('int({var}[0])',), 'uint16': ('int({var})',), ('uint16', '*'): ('int({var}[0])',), 'uint32': ('int({var})',), ('uint32', '*'): ('int({var}[0])',), 'uint64': ('int({var})',), 'float32': ('float({var})',), ('float32', '*'): ('float({var}[0])',), 'float64': ('float({var})',), ('float64', '*'): ('float({var}[0])',), 'float128': ('np.array({var}, dtype=np.float128)',), ('float128', '*'): ('np.array({var}[0], dtype=np.float128)',), 'complex128': ('complex(float({var}.re), float({var}.im))',), ('complex128', '*'): ('complex(float({var}[0].re), float({var}[0].im))',), 'bool': ('bool({var})',), ('bool', '*'): ('bool({var}[0])',), 'void': ('None',), 'file': ('{extra_types}PyFile_FromFile(&{var}, "{var}", "r+", NULL)',), ('file', '*'): ( '{extra_types}PyFile_FromFile({var}, "{var}", "r+", NULL)',), # template types 'map': ('{t.cython_pytype}({var})', ('{proxy_name} = {t.cython_pytype}(False, False)\n' '{proxy_name}.map_ptr = &{var}\n'), ('if {cache_name} is None:\n' ' {proxy_name} = {t.cython_pytype}(False, False)\n' ' {proxy_name}.map_ptr = &{var}\n' ' {cache_name} = {proxy_name}\n' )), 'dict': ('dict({var})',), 'pair': ('{t.cython_pytype}({var})', ('{proxy_name} = {t.cython_pytype}(False, False)\n' '{proxy_name}.pair_ptr = &{var}\n'), ('if {cache_name} is None:\n' ' {proxy_name} = {t.cython_pytype}(False, False)\n' ' {proxy_name}.pair_ptr = &{var}\n' ' {cache_name} = {proxy_name}\n' )), 'set': ('{t.cython_pytype}({var})', ('{proxy_name} = {t.cython_pytype}(False, False)\n' '{proxy_name}.set_ptr = &{var}\n'), ('if {cache_name} is None:\n' ' {proxy_name} = {t.cython_pytype}(False, False)\n' ' {proxy_name}.set_ptr = &{var}\n' ' {cache_name} = {proxy_name}\n' )), TypeMatcher(('set', MatchAny, '*')): ('{t.cython_pytype}(deref({var}))', ('{proxy_name} = {t.cython_pytype}(False, False)\n' '{proxy_name}.set_ptr = {var}\n'), ('if {cache_name} is None:\n' ' {proxy_name} = {t.cython_pytype}(False, False)\n' ' {proxy_name}.set_ptr = {var}\n' ' {cache_name} = {proxy_name}\n' )), 'vector': ( ('{proxy_name}_shape[0] = <np.npy_intp> {var}.size()\n' '{proxy_name} = np.PyArray_SimpleNewFromData(1, {var}_shape, {t.cython_nptypes[0]}, &{var}[0])\n' '{proxy_name} = np.PyArray_Copy({proxy_name})\n'), ('{proxy_name}_shape[0] = <np.npy_intp> {var}.size()\n' '{proxy_name} = np.PyArray_SimpleNewFromData(1, {proxy_name}_shape, {t.cython_nptypes[0]}, &{var}[0])\n'), ('if {cache_name} is None:\n' ' {proxy_name}_shape[0] = <np.npy_intp> {var}.size()\n' ' {proxy_name} = np.PyArray_SimpleNewFromData(1, {proxy_name}_shape, {t.cython_nptypes[0]}, &{var}[0])\n' ' {cache_name} = {proxy_name}\n' )), ('vector', 'bool', 0): ( # C++ standard is silly here ('cdef int i\n' '{proxy_name}_shape[0] = <np.npy_intp> {var}.size()\n' '{proxy_name} = np.PyArray_SimpleNew(1, {proxy_name}_shape, {t.cython_nptypes[0]})\n' 'for i in range({proxy_name}_shape[0]):\n' ' {proxy_name}[i] = {var}[i]\n'), ('cdef int i\n' '{proxy_name}_shape[0] = <np.npy_intp> {var}.size()\n' '{proxy_name} = np.PyArray_SimpleNew(1, {proxy_name}_shape, {t.cython_nptypes[0]})\n' 'for i in range({proxy_name}_shape[0]):\n' ' {proxy_name}[i] = {var}[i]\n'), ('cdef int i\n' 'if {cache_name} is None:\n' ' {proxy_name}_shape[0] = <np.npy_intp> {var}.size()\n' ' {proxy_name} = np.PyArray_SimpleNew(1, {proxy_name}_shape, {nptype[0]})\n' ' for i in range({proxy_name}_shape[0]):\n' ' {proxy_name}[i] = {var}[i]\n' ' {cache_name} = {proxy_name}\n' )), # C/C++ chars are ints while Python Chars are length-1 strings ('vector', 'char', 0): ( ('cdef int i\n' '{proxy_name}_shape[0] = <np.npy_intp> {var}.size()\n' '{proxy_name} = np.empty({proxy_name}_shape[0], "U1")\n' 'for i in range({proxy_name}_shape[0]):\n' ' {proxy_name}[i] = chr(<int> {var}[i])\n'), ('cdef int i\n' '{proxy_name}_shape[0] = <np.npy_intp> {var}.size()\n' '{proxy_name} = np.empty({proxy_name}_shape[0], "U1")\n' 'for i in range({proxy_name}_shape[0]):\n' ' {proxy_name}[i] = chr(<int> {var}[i])\n'), ('cdef int i\n' 'if {cache_name} is None:\n' ' {proxy_name}_shape[0] = <np.npy_intp> {var}.size()\n' ' for i in range({proxy_name}_shape[0]):\n' ' {proxy_name}[i] = chr(<int> {var}[i])\n' ' {cache_name} = {proxy_name}\n' )), 'nucid': ('nucname.zzaaam({var})',), 'nucname': ('nucname.name({var})',), TypeMatcher((('enum', MatchAny, MatchAny), '*')): ('int({var}[0])',), TypeMatcher((('int32', ('enum', MatchAny, MatchAny)), '*')): \ ('int({var}[0])',), # Strip const when going c -> py TypeMatcher((MatchAny, 'const')): ( lambda t, ts: ts.cython_c2py_getitem(t[0])), TypeMatcher(((MatchAny, 'const'), '&')) : ( lambda t, ts: ts.cython_c2py_getitem((t[0][0], '&'))), TypeMatcher(((MatchAny, 'const'), '*')): ( lambda t, ts: ts.cython_c2py_getitem((t[0][0], '*'))), 'function_pointer': cython_c2py_conv_function_pointer, }, self) cython_py2c_conv_vector_ref = (( '# {var} is a {t.type}\n' 'cdef int i{var}\n' 'cdef int {var}_size\n' 'cdef {t.cython_npctypes_nopred[0]} * {var}_data\n' '{var}_size = len({var})\n' 'if isinstance({var}, np.ndarray) and (<np.ndarray> {var}).descr.type_num == {t.cython_nptype}:\n' ' {var}_data = <{t.cython_npctypes_nopred[0]} *> np.PyArray_DATA(<np.ndarray> {var})\n' ' {proxy_name} = {t.cython_ctype_nopred}(<size_t> {var}_size)\n' ' for i{var} in range({var}_size):\n' ' {proxy_name}[i{var}] = {var}_data[i{var}]\n' 'else:\n' ' {proxy_name} = {t.cython_ctype_nopred}(<size_t> {var}_size)\n' ' for i{var} in range({var}_size):\n' ' {proxy_name}[i{var}] = <{t.cython_npctypes_nopred[0]}> {var}[i{var}]\n'), '{proxy_name}') # FIXME There might be improvements here... def cython_py2c_conv_function_pointer(t, ts): t = t[1] argnames = [] argcts = [] argdecls = [] argbodys = [] argrtns = [] for n, argt in t[1][2]: argnames.append(n) decl, body, rtn, _ = ts.cython_c2py(n, argt, proxy_name="c_" + n, cached=False) argdecls.append(decl) #argdecls.append("cdef {0} {1}".format(cython_pytype(argt), "c_" + n)) argbodys.append(body) argrtns.append(rtn) argct = ts.cython_ctype(argt) argcts.append(argct) rtnname = 'rtn' rtnprox = 'c_' + rtnname rtncall = 'call_' + rtnname while rtnname in argnames or rtnprox in argnames: rtnname += '_' rtnprox += '_' rtnct = ts.cython_ctype(t[2][2]) argdecls = indent(argdecls) argbodys = indent(argbodys) #rtndecl, rtnbody, rtnrtn = cython_py2c(rtnname, t[2][2], proxy_name=rtnprox) #rtndecl, rtnbody, rtnrtn = cython_py2c(rtnname, t[2][2], proxy_name=rtncall) rtndecl, rtnbody, rtnrtn = ts.cython_py2c(rtncall, t[2][2], proxy_name=rtnprox) if rtndecl is None and rtnbody is None: rtnprox = rtnname rtndecl = indent([rtndecl]) rtnbody = indent([rtnbody]) returns_void = (t[2][2] == 'void') if returns_void: rtnrtn = '' s = ('cdef {rtnct} {{proxy_name}}({arglist}):\n' '{argdecls}\n' '{rtndecl}\n' ' global {{var}}\n' '{argbodys}\n') s += ' {{var}}({pyarglist})\n' if returns_void else \ ' {rtncall} = {{var}}({pyarglist})\n' s += ('{rtnbody}\n' ' return {rtnrtn}\n') arglist = ", ".join(["{0} {1}".format(*x) for x in zip(argcts, argnames)]) pyarglist=", ".join(argrtns) s = s.format(rtnct=rtnct, arglist=arglist, argdecls=argdecls, rtndecl=rtndecl, argbodys=argbodys, rtnprox=rtnprox, pyarglist=pyarglist, rtnbody=rtnbody, rtnrtn=rtnrtn, rtncall=rtncall) return s, False self.cython_py2c_conv = _LazyConverterDict(cython_py2c_conv if \ cython_py2c_conv is not None else { # Has tuple form of (body or return, return or False) # base types 'char': ('{var}_bytes = {var}.encode()', '(<char *> {var}_bytes)[0]'), ('char', '*'): ('{var}_bytes = {var}.encode()', '<char *> {var}_bytes'), (('char', '*'), '*'): ('cdef char * {var}_bytes_\n' '{var}_bytes = {var}[0].encode()\n' '{var}_bytes_ = {var}_bytes\n' '{proxy_name} = &{var}_bytes_', '{proxy_name}'), 'uchar': ('{var}_bytes = {var}.encode()', '(<unsigned char *> {var}_bytes)[0]'), ('uchar', '*'): ('{var}_bytes = {var}.encode()', '<unsigned char *> {var}_bytes'), 'str': ('{var}_bytes = {var}.encode()', 'std_string(<char *> {var}_bytes)'), 'int16': ('<short> {var}', False), 'int32': ('<int> {var}', False), #('int32', '*'): ('&(<int> {var})', False), ('int32', '*'): ('cdef int {proxy_name}_ = {var}', '&{proxy_name}_'), 'int64': ('<long long> {var}', False), 'uint16': ('<unsigned short> {var}', False), 'uint32': ('<{t.cython_ctype}> long({var})', False), #'uint32': ('<unsigned long> {var}', False), #('uint32', '*'): ('cdef unsigned long {proxy_name}_ = {var}', '&{proxy_name}_'), ('uint32', '*'): ('cdef unsigned int {proxy_name}_ = {var}', '&{proxy_name}_'), 'uint64': ('<unsigned long long> {var}', False), 'float32': ('<float> {var}', False), 'float64': ('<double> {var}', False), ('float64', '*'): ('cdef double {proxy_name}_ = {var}', '&{proxy_name}_'), 'float128': ('<long double> {var}', False), 'complex128': ('{extra_types}py2c_complex({var})', False), 'bool': ('<bint> {var}', False), 'void': ('NULL', False), ('void', '*'): ('NULL', False), 'file': ('{extra_types}PyFile_AsFile({var})[0]', False), ('file', '*'): ('{extra_types}PyFile_AsFile({var})', False), # template types 'map': ('{proxy_name} = {t.cython_pytype}({var}, not isinstance({var}, {t.cython_cytype}))', '{proxy_name}.map_ptr[0]'), 'dict': ('dict({var})', False), 'pair': ('{proxy_name} = {t.cython_pytype}({var}, not isinstance({var}, {t.cython_cytype}))', '{proxy_name}.pair_ptr[0]'), 'set': ('{proxy_name} = {t.cython_pytype}({var}, not isinstance({var}, {t.cython_cytype}))', '{proxy_name}.set_ptr[0]'), 'vector': (( '# {var} is a {t.type}\n' 'cdef int i{var}\n' 'cdef int {var}_size\n' 'cdef {t.cython_npctypes[0]} * {var}_data\n' '{var}_size = len({var})\n' 'if isinstance({var}, np.ndarray) and (<np.ndarray> {var}).descr.type_num == {t.cython_nptype}:\n' ' {var}_data = <{t.cython_npctypes[0]} *> np.PyArray_DATA(<np.ndarray> {var})\n' ' {proxy_name} = {t.cython_ctype}(<size_t> {var}_size)\n' ' for i{var} in range({var}_size):\n' ' {proxy_name}[i{var}] = {var}_data[i{var}]\n' 'else:\n' ' {proxy_name} = {t.cython_ctype}(<size_t> {var}_size)\n' ' for i{var} in range({var}_size):\n' ' {proxy_name}[i{var}] = <{t.cython_npctypes[0]}> {var}[i{var}]\n'), '{proxy_name}'), # FIXME There might be improvements here... ('vector', 'char', 0): (( '# {var} is a {t.type}\n' 'cdef int i{var}\n' 'cdef int {var}_size\n' 'cdef {t.cython_npctypes[0]} * {var}_data\n' '{var}_size = len({var})\n' 'if isinstance({var}, np.ndarray) and (<np.ndarray> {var}).descr.type_num == <int> {t.cython_nptype}:\n' ' {var}_data = <{t.cython_npctypes[0]} *> np.PyArray_DATA(<np.ndarray> {var})\n' ' {proxy_name} = {t.cython_ctype}(<size_t> {var}_size)\n' ' for i{var} in range({var}_size):\n' ' {proxy_name}[i{var}] = {var}[i{var}]\n' 'else:\n' ' {proxy_name} = {t.cython_ctype}(<size_t> {var}_size)\n' ' for i{var} in range({var}_size):\n' ' _ = {var}[i{var}].encode()\n' ' {proxy_name}[i{var}] = deref(<char *> _)\n'), '{proxy_name}'), TypeMatcher(('vector', MatchAny, '&')): cython_py2c_conv_vector_ref, TypeMatcher((('vector', MatchAny, 0), '&')): cython_py2c_conv_vector_ref, TypeMatcher((('vector', MatchAny, '&'), 0)): cython_py2c_conv_vector_ref, TypeMatcher((('vector', MatchAny, '&'), 'const')): cython_py2c_conv_vector_ref, TypeMatcher((('vector', MatchAny, 'const'), '&')): cython_py2c_conv_vector_ref, TypeMatcher(((('vector', MatchAny, 0), 'const'), '&')): cython_py2c_conv_vector_ref, TypeMatcher(((('vector', MatchAny, 0), '&'), 'const')): cython_py2c_conv_vector_ref, # refinement types 'nucid': ('nucname.zzaaam({var})', False), 'nucname': ('nucname.name({var})', False), TypeMatcher((('enum', MatchAny, MatchAny), '*')): \ ('cdef int {proxy_name}_ = {var}', '&{proxy_name}_'), TypeMatcher((('int32', ('enum', MatchAny, MatchAny)), '*')): \ ('cdef int {proxy_name}_ = {var}', '&{proxy_name}_'), 'function_pointer': cython_py2c_conv_function_pointer, }, self) self.typestr = typestring or typestr @classmethod
[docs] def empty(cls): """This is a class method which returns an empty type system.""" x = cls(base_types=set(), template_types={}, refined_types={}, humannames={}, type_aliases={}, cpp_types={}, numpy_types={}, from_pytypes={}, cython_ctypes={}, cython_cytypes={}, cython_pytypes={}, cython_cimports={}, cython_cyimports={}, cython_pyimports={}, cython_functionnames={}, cython_classnames={}, cython_c2py_conv={}, cython_py2c_conv={}) del x.extra_types del x.dtypes del x.stlcontainers return x
@classmethod
[docs] def load(cls, filename, format=None, mode='rb'): """Loads a type system from disk into a new type system instance. This is a class method. Parameters ---------- filename : str Path to file. format : str, optional The file format to save the type system as. If this is not provided, it is infered from the filenme. Options are: * pickle ('.pkl') * gzipped pickle ('.pkl.gz') mode : str, optional The mode to open the file with. """ format = infer_format(filename, format) if not os.path.isfile(filename): raise RuntimeError("{0!r} not found.".format(filename)) if format == 'pkl.gz': f = gzip.open(filename, 'rb') data = pickle.loads(f.read()) f.close() elif format == 'pkl': with io.open(filename, 'rb') as f: data = pickle.loads(f.read()) x = cls(**data) return x
[docs] def dump(self, filename, format=None, mode='wb'): """Saves a type system out to disk. Parameters ---------- filename : str Path to file. format : str, optional The file format to save the type system as. If this is not provided, it is infered from the filenme. Options are: * pickle ('.pkl') * gzipped pickle ('.pkl.gz') mode : str, optional The mode to open the file with. """ data = dict([(k, getattr(self, k, None)) for k in self.datafields]) format = infer_format(filename, format) if format == 'pkl.gz': f = gzip.open(filename, mode) f.write(pickle.dumps(data, pickle.HIGHEST_PROTOCOL)) f.close() elif format == 'pkl': with io.open(filename, mode) as f: f.write(pickle.dumps(data, pickle.HIGHEST_PROTOCOL))
[docs] def update(self, *args, **kwargs): """Updates the type system in-place. Only updates the data attributes named in 'datafields'. This may be called with any of the following signatures:: ts.update(<TypeSystem>) ts.update(<dict-like>) ts.update(key1=value1, key2=value2, ...) Valid keyword arguments are the same here as for the type system constructor. See this documentation for more detail. """ datafields = self.datafields # normalize arguments if len(args) == 1 and len(kwargs) == 0: toup = args[0] if isinstance(toup, TypeSystem): toup = dict([(k, getattr(toup, k)) for k in datafields \ if hasattr(toup, k)]) elif not isinstance(toup, Mapping): toup = dict(toup) elif len(args) == 0: toup = kwargs else: msg = "invalid siganture: args={0!r}, kwargs={1!0}" raise TypeError(msg.fomat(args, kwargs)) # verify keys for k in toup: if k not in datafields: msg = "{0} is not a member of {1}" raise AttributeError(msg.format(k, self.__class__.__name__)) # perform the update for k, v in toup.items(): x = getattr(self, k) if isinstance(v, Mapping): x.update(v) elif isinstance(v, Set): x.update(v) else: setattr(self, k, v)
def __str__(self): s = pformat(dict([(k, getattr(self, k, None)) for k in \ sorted(self.datafields)])) return s def __repr__(self): s = self.__class__.__name__ + "(" s += ", ".join(["{0}={1!r}".format(k, getattr(self, k, None)) \ for k in sorted(self.datafields)]) s += ")" return s #################### Important Methods below ############################### @memoize_method
[docs] def istemplate(self, t): """Returns whether t is a template type or not.""" if isinstance(t, basestring): return t in self.template_types if isinstance(t, Sequence): return self.istemplate(t[0]) return False
@memoize_method def isenum(self, t): try: t = self.canon(t) except TypeError: return False return isinstance(t, Sequence) and t[0] == 'int32' and \ isinstance(t[1], Sequence) and t[1][0] == 'enum' @memoize_method def isfunctionpointer(self, t): t = self.canon(t) return isinstance(t, Sequence) and t[0] == ('void', '*') and \ isinstance(t[1], Sequence) and t[1][0] == 'function_pointer' @memoize_method
[docs] def humanname(self, t, hnt=None): """Computes human names for types.""" if hnt is None: t = self.canon(t) if isinstance(t, basestring): return t, self.humannames[t] elif t[0] in self.base_types: return t, self.humannames[t[0]] return self.humanname(t, self.humannames[t[0]]) d = {} for key, x in zip(self.template_types[t[0]], t[1:-1]): if isinstance(x, basestring): val = self.humannames[x] elif isinstance(x, int): val = x elif x[0] in self.base_types: val = self.humannames[x[0]] else: val, _ = self.humanname(x, self.humannames[x[0]]) d[key] = val return t, hnt.format(**d)
@memoize_method
[docs] def isdependent(self, t): """Returns whether t is a dependent type or not.""" deptypes = set([k[0] for k in self.refined_types \ if not isinstance(k, basestring)]) if isinstance(t, basestring): return t in deptypes if isinstance(t, Sequence): return self.isdependent(t[0]) return False
@memoize_method
[docs] def isrefinement(self, t): """Returns whether t is a refined type.""" if isinstance(t, basestring): return t in self.refined_types return self.isdependent(t)
@memoize_method def _resolve_dependent_type(self, tname, tinst=None): depkey = [k for k in self.refined_types if k[0] == tname][0] depval = self.refined_types[depkey] istemplated = self.istemplate(depkey) if tinst is None: return depkey elif istemplated: assert len(tinst) == len(depkey) typemap = dict([(k, tinst[i]) for i, k in enumerate(depkey[1:], 1) \ if isinstance(k, basestring)]) for k in typemap: if k in self.type_aliases: raise TypeError('template type {0} already exists'.format(k)) self.type_aliases.update(typemap) resotype = self.canon(depval), (tname,) + \ tuple([self.canon(k) for k in depkey[1:] if k in typemap]) + \ tuple([(k[0], self.canon(k[1]), instval) \ for k, instval in zip(depkey[1:], tinst[1:]) if k not in typemap]) for k in typemap: del self.type_aliases[k] self.delmemo('canon', k) return resotype else: assert len(tinst) == len(depkey) return self.canon(depval), (tname,) + tuple([(kname, self.canon(ktype), instval) for (kname, ktype), instval in zip(depkey[1:], tinst[1:])]) @memoize_method
[docs] def canon(self, t): """Turns the type into its canonical form. See module docs for more information.""" if isinstance(t, basestring): if t in self.base_types: return t elif t in self.type_aliases: return self.canon(self.type_aliases[t]) elif t in self.refined_types: return (self.canon(self.refined_types[t]), t) elif self.isdependent(t): return self._resolve_dependent_type(t) else: _raise_type_error(t) # BELOW this would be for complicated string representations, # such as 'char *' or 'map<nucid, double>'. Would need to write # the parse_type() function and that might be a lot of work. #parse_type(t) elif isinstance(t, Sequence): t0 = t[0] tlen = len(t) if 0 == tlen: _raise_type_error(t) last_val = 0 if tlen == 1 else t[-1] if not isinstance(t0, basestring) and not isinstance(t0, Sequence): _raise_type_error(t) if self.isdependent(t0): return self._resolve_dependent_type(t0, t) elif t0 in self.template_types: templen = len(self.template_types[t0]) last_val = 0 if tlen == 1 + templen else t[-1] filledt = [t0] for tt in t[1:1+templen]: if isinstance(tt, Number): # includes bool! filledt.append(tt) elif isinstance(tt, basestring): try: canontt = self.canon(tt) except TypeError: canontt = tt except: raise filledt.append(canontt) elif isinstance(tt, Sequence): if len(tt) == 2 and tt[0] in Arg: filledt.append(self.canon(tt[1])) else: filledt.append(self.canon(tt)) else: _raise_type_error(tt) filledt.append(last_val) return tuple(filledt) else: return (self.canon(t0), last_val) else: _raise_type_error(t)
@memoize_method
[docs] def strip_predicates(self, t): """Removes all outer predicates from a type.""" t = self.canon(t) if isinstance(t, basestring): return t elif isinstance(t, Sequence): tlen = len(t) if tlen == 2: sp0 = self.strip_predicates(t[0]) return (sp0, 0) if t[1] == 0 else sp0 else: return t[:-1] + (0,) else: _raise_type_error(t)
@memoize_method
[docs] def basename(self, t): """Retrieves basename from a type, e.g. 'map' in ('map', 'int', 'float').""" t = self.canon(t) if isinstance(t, basestring): return t elif isinstance(t, Sequence): t0 = t while not isinstance(t0, basestring): t0 = t0[0] return t0 else: _raise_type_error(t) ########################### C/C++ Methods #############################
def _cpp_type_add_predicate(self, t, last): """Adds a predicate to a C++ type""" if last == 'const': x, y = last, t else: x, y = t, last return '{0} {1}'.format(x, y) def _cpp_var_name(self, var): ns = self.variable_namespace.get(var, '') if len(ns) > 0: name = ns + "::" + var else: name = var return name @memoize_method
[docs] def cpp_type(self, t): """Given a type t, returns the corresponding C++ type declaration.""" t = self.canon(t) if isinstance(t, basestring): if t in self.base_types: return self.cpp_types[t] # must be tuple below this line tlen = len(t) if 2 == tlen: if 0 == t[1]: return self.cpp_type(t[0]) elif self.isrefinement(t[1]): if t[1][0] in self.cpp_types: subtype = self.cpp_types[t[1][0]] if callable(subtype): subtype = subtype(t[1], self) return subtype else: return self.cpp_type(t[0]) else: last = '[{0}]'.format(t[-1]) if isinstance(t[-1], int) else t[-1] return self._cpp_type_add_predicate(self.cpp_type(t[0]), last) elif 3 <= tlen: assert t[0] in self.template_types assert len(t) == len(self.template_types[t[0]]) + 2 template_name = self.cpp_types[t[0]] assert template_name is not NotImplemented template_filling = [] kinds = self.argument_kinds.get(t, ((Arg.NONE,),)*(tlen-2)) for x, kind in zip(t[1:-1], kinds): if kind is Arg.LIT: x = self.cpp_literal(x) elif kind is Arg.TYPE: x = self.cpp_type(x) elif kind is Arg.VAR: x = self._cpp_var_name(x) elif isinstance(x, bool): x = self.cpp_types[x] elif isinstance(x, Number): x = str(x) else: try: x = self.cpp_type(x) # Guess it is a type? except TypeError: x = self._cpp_var_name(x) # Guess it is a variable template_filling.append(x) cppt = '{0}< {1} >'.format(template_name, ', '.join(template_filling)) if 0 != t[-1]: last = '[{0}]'.format(t[-1]) if isinstance(t[-1], int) else t[-1] cppt = self._cpp_type_add_predicate(cppt, last) return cppt
@memoize_method
[docs] def cpp_literal(self, lit): """Converts a literal value to it C++ form. """ if isinstance(lit, bool): cpp_lit = self.cpp_types[lit] elif isinstance(lit, Number): cpp_lit = str(lit) elif isinstance(lit, basestring): cpp_lit = repr(lit) return cpp_lit
@memoize_method
[docs] def cpp_funcname(self, name, argkinds=None): """This returns a name for a function based on its name, rather than its type. The name may be either a string or a tuple of the form ('name', template_arg1, template_arg2, ...). The argkinds argument here refers only to the template arguments, not the function signature default arguments. This is not meant to replace cpp_type(), but complement it. """ if isinstance(name, basestring): return name if argkinds is None: argkinds = [(Arg.NONE, None)] * (len(name) - 1) fname = name[0] cts = [] for x, (argkind, argvalue) in zip(name[1:], argkinds): if argkind is Arg.TYPE: ct = self.cpp_type(x) elif argkind is Arg.LIT: ct = self.cpp_literal(x) elif isinstance(x, Number): ct = self.cpp_literal(x) else: try: ct = self.cpp_type(x) # guess it is a type except TypeError: ct = x # guess it is a variable cts.append(ct) fname += '' if 0 == len(cts) else "< " + ", ".join(cts) + " >" return fname
@memoize_method
[docs] def gccxml_type(self, t): """Given a type t, returns the corresponding GCC-XML type name.""" cppt = self.cpp_type(t) gxt = cppt.replace('< ', '<').replace(' >', '>').\ replace('>>', '> >').replace(', ', ',') return gxt
@memoize_method
[docs] def cython_nptype(self, t, depth=0): """Given a type t, returns the corresponding numpy type. If depth is greater than 0 then this returns of a list of numpy types for all internal template types, ie the float in ('vector', 'float', 0). """ if isinstance(t, Number): return 'np.NPY_OBJECT' t = self.canon(t) if isinstance(t, basestring): return self.numpy_types[t] if t in self.numpy_types else 'np.NPY_OBJECT' # must be tuple below this line tlen = len(t) if t in self.numpy_types and depth < 1: return self.numpy_types[t] elif 2 == tlen: if 0 == t[1]: return self.cython_nptype(t[0]) elif self.isrefinement(t[1]): return self.cython_nptype(t[0]) else: # FIXME last is ignored for strings, but what about other types? #last = '[{0}]'.format(t[-1]) if isinstance(t[-1], int) else t[-1] #return cython_pytype(t[0]) + ' {0}'.format(last) return self.cython_nptype(t[0]) elif 0 < depth and self.istemplate(t): depth -= 1 return [self.cython_nptype(u, depth=depth) for u in t[1:-1]] elif 3 == tlen and self.istemplate(t): return self.cython_nptype(t[1]) else: # elif 3 <= tlen: return 'np.NPY_OBJECT' ######################### Cython Functions ############################
def _cython_ctype_add_predicate(self, t, last): """Adds a predicate to a ctype""" if last == 'const': x, y = last, t else: x, y = t, last return '{0} {1}'.format(x, y) @memoize_method
[docs] def cython_ctype(self, t): """Given a type t, returns the corresponding Cython C/C++ type declaration. """ t = self.canon(t) if t in self.cython_ctypes: return self.cython_ctypes[t] if isinstance(t, basestring): if t in self.base_types: return self.cython_ctypes[t] # must be tuple below this line tlen = len(t) if 2 == tlen: if 0 == t[1]: return self.cython_ctype(t[0]) elif self.isrefinement(t[1]): if t[1][0] in self.cython_ctypes: subtype = self.cython_ctypes[t[1][0]] if callable(subtype): subtype = subtype(t[1], self) return subtype else: return self.cython_ctype(t[0]) else: last = '[{0}]'.format(t[-1]) if isinstance(t[-1], int) else t[-1] return self._cython_ctype_add_predicate(self.cython_ctype(t[0]), last) elif 3 <= tlen: assert t[0] in self.template_types assert len(t) == len(self.template_types[t[0]]) + 2 template_name = self.cython_ctypes[t[0]] assert template_name is not NotImplemented template_filling = [] for x in t[1:-1]: #if isinstance(x, bool): # x = _cython_ctypes[x] #elif isinstance(x, Number): if isinstance(x, Number): x = str(x) else: x = self.cython_ctype(x) template_filling.append(x) cyct = '{0}[{1}]'.format(template_name, ', '.join(template_filling)) if 0 != t[-1]: last = '[{0}]'.format(t[-1]) if isinstance(t[-1], int) else t[-1] cyct = self._cython_ctype_add_predicate(cyct, last) return cyct
@memoize_method def _fill_cycyt(self, cycyt, t): """Helper for cython_cytype().""" d = {} for key, x in zip(self.template_types[t[0]], t[1:-1]): if isinstance(x, basestring): val = self.cython_classnames[x] elif isinstance(x, Number): val = str(x) elif x[0] in self.base_types: val = self.cython_classnames[x[0]] else: val, _ = self._fill_cycyt(self.cython_classnames[x[0]], x) d[key] = val return cycyt.format(**d), t def _cython_cytype_add_predicate(self, t, last): """Adds a predicate to a cytype""" if last == '*': return '{0} {1}'.format(t, last) elif isinstance(last, int) and 0 < last: return '{0} [{1}]'.format(t, last) else: return t @memoize_method
[docs] def cython_cytype(self, t): """Given a type t, returns the corresponding Cython type.""" t = self.canon(t) # if t in self.cython_cytypes: # return self.cython_cytypes[t] if isinstance(t, basestring): if t in self.base_types or t in self.cython_cytypes: return self.cython_cytypes[t] # must be tuple below this line tlen = len(t) if 2 == tlen: if 0 == t[1]: return self.cython_cytype(t[0]) elif self.isrefinement(t[1]): if t[1][0] in self.cython_cytypes: subtype = self.cython_cytypes[t[1][0]] if callable(subtype): subtype = subtype(t[1], self) return subtype else: return self.cython_cytype(t[0]) else: return self._cython_cytype_add_predicate(self.cython_cytype(t[0]), t[-1]) elif 3 <= tlen: if t in self.cython_cytypes: return self.cython_cytypes[t] assert t[0] in self.template_types assert len(t) == len(self.template_types[t[0]]) + 2 template_name = self.cython_cytypes[t[0]] assert template_name is not NotImplemented cycyt = self.cython_cytypes[t[0]] cycyt, t = self._fill_cycyt(cycyt, t) cycyt = self._cython_cytype_add_predicate(cycyt, t[-1]) return cycyt
@memoize_method def _fill_cypyt(self, cypyt, t): """Helper for cython_pytype().""" d = {} for key, x in zip(self.template_types[t[0]], t[1:-1]): if isinstance(x, basestring): val = self.cython_classnames[x] elif isinstance(x, Number): val = str(x) elif x[0] in self.base_types: val = self.cython_classnames[x[0]] else: val, _ = self._fill_cypyt(self.cython_classnames[x[0]], x) d[key] = val return cypyt.format(**d), t @memoize_method
[docs] def cython_pytype(self, t): """Given a type t, returns the corresponding Python type.""" if isinstance(t, Number): return str(t) t = self.canon(t) if t in self.cython_pytypes: return self.cython_pytypes[t] if isinstance(t, basestring): if t in self.base_types: return self.cython_pytypes[t] # must be tuple below this line tlen = len(t) if 2 == tlen: if 0 == t[1]: return self.cython_pytype(t[0]) elif self.isrefinement(t[1]): return self.cython_pytype(t[0]) else: # FIXME last is ignored for strings, but what about other types? #last = '[{0}]'.format(t[-1]) if isinstance(t[-1], int) else t[-1] #return cython_pytype(t[0]) + ' {0}'.format(last) return self.cython_pytype(t[0]) elif 3 <= tlen: if t in self.cython_pytypes: return self.cython_pytypes[t] assert t[0] in self.template_types assert len(t) == len(self.template_types[t[0]]) + 2 template_name = self.cython_pytypes[t[0]] assert template_name is not NotImplemented cypyt = self.cython_pytypes[t[0]] cypyt, t = self._fill_cypyt(cypyt, t) # FIXME last is ignored for strings, but what about other types? #if 0 != t[-1]: # last = '[{0}]'.format(t[-1]) if isinstance(t[-1], int) else t[-1] # cypyt += ' {0}'.format(last) return cypyt
@memoize_method
[docs] def cython_cimport_tuples(self, t, seen=None, inc=frozenset(['c', 'cy'])): """Given a type t, and possibly previously seen cimport tuples (set), return the set of all seen cimport tuples. These tuple have four possible interpretations based on the length and values: * ``(module-name,)`` becomes ``cimport {module-name}`` * ``(module-name, var-or-mod)`` becomes ``from {module-name} cimport {var-or-mod}`` * ``(module-name, var-or-mod, alias)`` becomes ``from {module-name} cimport {var-or-mod} as {alias}`` * ``(module-name, 'as', alias)`` becomes ``cimport {module-name} as {alias}`` """ t = self.canon(t) if seen is None: seen = set() if isinstance(t, basestring): if t in self.base_types: if 'c' in inc: seen.update(self.cython_cimports[t]) if 'cy' in inc: seen.update(self.cython_cyimports[t]) seen -= set((None, (None,))) return seen # must be tuple below this line tlen = len(t) if 2 == tlen: if 'c' in inc: if self.isrefinement(t[1]) and t[1][0] in self.cython_cimports: f = self.cython_cimports[t[1][0]] if callable(f): f(t[1], self, seen) seen.update(self.cython_cimports.get(t[0], (None,))) seen.update(self.cython_cimports.get(t[1], (None,))) if 'cy' in inc: if self.isrefinement(t[1]) and t[1][0] in self.cython_cyimports: f = self.cython_cyimports[t[1][0]] if callable(f): f(t[1], self, seen) seen.update(self.cython_cyimports.get(t[0], (None,))) seen.update(self.cython_cyimports.get(t[1], (None,))) seen -= set((None, (None,))) return self.cython_cimport_tuples(t[0], seen, inc) elif 3 <= tlen: assert t[0] in self.template_types if 'c' in inc: seen.update(self.cython_cimports[t[0]]) if 'cy' in inc: seen.update(self.cython_cyimports[t[0]]) for x in t[1:-1]: if isinstance(x, Number): continue elif isinstance(x, basestring) and x not in self.cython_cimports: continue self.cython_cimport_tuples(x, seen, inc) seen -= set((None, (None,))) return seen
_cython_cimport_cases = { 1: lambda tup: "cimport {0}".format(*tup), 2: lambda tup: "from {0} cimport {1}".format(*tup), 3: lambda tup: ("cimport {0} as {2}".format(*tup) if tup[1] == 'as' else \ "from {0} cimport {1} as {2}".format(*tup)), } @memoize_method
[docs] def cython_cimport_lines(self, x, inc=frozenset(['c', 'cy'])): """Returns the cimport lines associated with a type or a set of seen tuples. """ if not isinstance(x, Set): x = self.cython_cimport_tuples(x, inc=inc) return set([self._cython_cimport_cases[len(tup)](tup) for tup in x \ if 0 != len(tup)])
@memoize_method
[docs] def cython_import_tuples(self, t, seen=None): """Given a type t, and possibly previously seen import tuples (set), return the set of all seen import tuples. These tuple have four possible interpretations based on the length and values: * ``(module-name,)`` becomes ``import {module-name}`` * ``(module-name, var-or-mod)`` becomes ``from {module-name} import {var-or-mod}`` * ``(module-name, var-or-mod, alias)`` becomes ``from {module-name} import {var-or-mod} as {alias}`` * ``(module-name, 'as', alias)`` becomes ``import {module-name} as {alias}`` Any of these may be used. """ t = self.canon(t) if seen is None: seen = set() if isinstance(t, basestring): if t in self.base_types: seen.update(self.cython_pyimports[t]) seen -= set((None, (None,))) return seen # must be tuple below this line tlen = len(t) if 2 == tlen: if self.isrefinement(t[1]) and t[1][0] in self.cython_pyimports: f = self.cython_pyimports[t[1][0]] if callable(f): f(t[1], self, seen) seen.update(self.cython_pyimports.get(t[0], (None,))) seen.update(self.cython_pyimports.get(t[1], (None,))) seen -= set((None, (None,))) return self.cython_import_tuples(t[0], seen) elif 3 <= tlen: assert t[0] in self.template_types seen.update(self.cython_pyimports[t[0]]) for x in t[1:-1]: if isinstance(x, Number): continue elif isinstance(x, basestring) and x not in self.cython_cimports: continue self.cython_import_tuples(x, seen) seen -= set((None, (None,))) return seen
_cython_import_cases = { 1: lambda tup: "import {0}".format(*tup), 2: lambda tup: "from {0} import {1}".format(*tup), 3: lambda tup: ("import {0} as {2}".format(*tup) if tup[1] == 'as' else \ "from {0} import {1} as {2}".format(*tup)), } @memoize_method
[docs] def cython_import_lines(self, x): """Returns the import lines associated with a type or a set of seen tuples. """ if not isinstance(x, Set): x = self.cython_import_tuples(x) x = [tup for tup in x if 0 < len(tup)] return set([self._cython_import_cases[len(tup)](tup) for tup in x])
@memoize_method
[docs] def cython_literal(self, lit): """Converts a literal to a Cython compatible form. """ if isinstance(lit, Number): cy_lit = str(lit).replace('-', 'Neg').replace('+', 'Pos')\ .replace('.', 'point') elif isinstance(lit, basestring): cy_lit = repr(lit) return cy_lit
@memoize_method
[docs] def cython_funcname(self, name, argkinds=None): """This returns a name for a function based on its name, rather than its type. The name may be either a string or a tuple of the form ('name', template_arg1, template_arg2, ...). The argkinds argument here refers only to the template arguments, not the function signature default arguments. This is not meant to replace cython_functionname(), but complement it. """ if isinstance(name, basestring): return name if argkinds is None: argkinds = [(Arg.NONE, None)] * (len(name) - 1) fname = name[0] cfs = [] for x, (argkind, argvalue) in zip(name[1:], argkinds): if argkind is Arg.TYPE: cf = self.cython_functionname(x)[1] elif argkind is Arg.LIT: cf = self.cython_literal(x) elif argkind is Arg.VAR: cf = x elif isinstance(x, Number): cf = self.cython_literal(x) else: try: cf = self.cython_functionname(x)[1] # guess type except TypeError: cf = x # guess variable cfs.append(cf) fname += '' if 0 == len(cfs) else "_" + "_".join(cfs) return fname
@memoize_method
[docs] def cython_functionname(self, t, cycyt=None): """Computes variable or function names for cython types.""" if cycyt is None: t = self.canon(t) if isinstance(t, basestring): return t, self.cython_functionnames[t] elif t[0] in self.base_types: return t, self.cython_functionnames[t[0]] return self.cython_functionname(t, self.cython_functionnames[t[0]]) d = {} for key, x in zip(self.template_types[t[0]], t[1:-1]): if isinstance(x, basestring): val = self.cython_functionnames[x] if x in self.cython_functionnames \ else x elif isinstance(x, Number): val = str(x).replace('-', 'Neg').replace('+', 'Pos')\ .replace('.', 'point') elif x[0] in self.base_types: val = self.cython_functionnames[x[0]] else: _, val = self.cython_functionname(x, self.cython_functionnames[x[0]]) d[key] = val return t, cycyt.format(**d)
cython_variablename = cython_functionname @memoize_method
[docs] def cython_classname(self, t, cycyt=None): """Computes classnames for cython types.""" if cycyt is None: t = self.canon(t) if isinstance(t, basestring): return t, self.cython_classnames[t] elif t[0] in self.base_types: return t, self.cython_classnames[t[0]] return self.cython_classname(t, self.cython_classnames[t[0]]) d = {} for key, x in zip(self.template_types[t[0]], t[1:-1]): if isinstance(x, basestring): val = self.cython_classnames[x] if x in self.cython_classnames else x elif isinstance(x, Number): val = str(x).replace('-', 'Neg').replace('+', 'Pos')\ .replace('.', 'point') elif x[0] in self.base_types: val = self.cython_classnames[x[0]] else: _, val = self.cython_classname(x, self.cython_classnames[x[0]]) d[key] = val return t, cycyt.format(**d)
@memoize_method
[docs] def cython_c2py_getitem(self, t): """Helps find the approriate c2py value for a given concrete type key.""" tkey = t = self.canon(t) while tkey not in self.cython_c2py_conv and not isinstance(tkey, basestring): #tkey = tkey[0] tkey = tkey[1] if (0 < len(tkey) and self.isrefinement(tkey[1])) else \ tkey[0] if tkey not in self.cython_c2py_conv: tkey = t while tkey not in self.cython_c2py_conv and \ not isinstance(tkey, basestring): tkey = tkey[0] c2pyt = self.cython_c2py_conv[tkey] if callable(c2pyt): self.cython_c2py_conv[t] = c2pyt(t, self) c2pyt = self.cython_c2py_conv[t] return c2pyt
@memoize_method
[docs] def cython_c2py(self, name, t, view=True, cached=True, inst_name=None, proxy_name=None, cache_name=None, cache_prefix='self', existing_name=None): """Given a varibale name and type, returns cython code (declaration, body, and return statements) to convert the variable from C/C++ to Python.""" t = self.canon(t) c2pyt = self.cython_c2py_getitem(t) ind = int(view) + int(cached) if cached and not view: raise ValueError('cached views require view=True.') if c2pyt is NotImplemented: raise NotImplementedError('conversion from C/C++ to Python for ' + \ t + 'has not been implemented for when ' + \ 'view={0}, cached={1}'.format(view, cached)) var = name if inst_name is None else "{0}.{1}".format(inst_name, name) var = existing_name or var cache_name = "_{0}".format(name) if cache_name is None else cache_name cache_name = cache_name if cache_prefix is None else "{0}.{1}".format( cache_prefix, cache_name) proxy_name = "{0}_proxy".format(name) if proxy_name is None else proxy_name iscached = False tstr = self.typestr(t, self) template_kw = dict(var=var, cache_name=cache_name, proxy_name=proxy_name, t=tstr) # if callable(c2pyt): # import pdb; pdb.set_trace() if 1 == len(c2pyt) or ind == 0: decl = body = None rtn = c2pyt[0].format(**template_kw) elif ind == 1: decl = "cdef {0} {1}".format(tstr.cython_cytype, proxy_name) body = c2pyt[1].format(**template_kw) rtn = proxy_name elif ind == 2: decl = "cdef {0} {1}".format(tstr.cython_cytype, proxy_name) body = c2pyt[2].format(**template_kw) rtn = cache_name iscached = True if body is not None and 'np.npy_intp' in body: decl = decl or '' decl += "\ncdef np.npy_intp {proxy_name}_shape[1]".format( proxy_name=proxy_name) if decl is not None and body is not None: newdecl = '\n'+"\n".join([l for l in body.splitlines() \ if l.startswith('cdef')]) body = "\n".join([l for l in body.splitlines() \ if not l.startswith('cdef')]) proxy_in_newdecl = proxy_name in [l.split()[-1] for l in \ newdecl.splitlines() if 0 < len(l)] if proxy_in_newdecl: for d in decl.splitlines(): if d.split()[-1] != proxy_name: newdecl += '\n' + d decl = newdecl else: decl += newdecl return decl, body, rtn, iscached
@memoize_method
[docs] def cython_py2c(self, name, t, inst_name=None, proxy_name=None): """Given a varibale name and type, returns cython code (declaration, body, and return statement) to convert the variable from Python to C/C++.""" t = self.canon(t) if isinstance(t, basestring) or 0 == t[-1] or self.isrefinement(t[-1]): last = '' elif isinstance(t[-1], int): last = ' [{0}]'.format(t[-1]) else: last = ' ' + t[-1] tkey = t tinst = None while tkey not in self.cython_py2c_conv and not isinstance(tkey, basestring): tinst = tkey tkey = tkey[1] if (0 < len(tkey) and self.isrefinement(tkey[1])) else tkey[0] if tkey not in self.cython_py2c_conv: tkey = t while tkey not in self.cython_py2c_conv and \ not isinstance(tkey, basestring): tkey = tkey[0] py2ct = self.cython_py2c_conv[tkey] if callable(py2ct): self.cython_py2c_conv[t] = py2ct(t, self) py2ct = self.cython_py2c_conv[t] if py2ct is NotImplemented or py2ct is None: raise NotImplementedError('conversion from Python to C/C++ for ' + \ str(t) + ' has not been implemented.') body_template, rtn_template = py2ct var = name if inst_name is None else "{0}.{1}".format(inst_name, name) proxy_name = "{0}_proxy".format(name) if proxy_name is None else proxy_name tstr = self.typestr(t, self) template_kw = dict(var=var, proxy_name=proxy_name, last=last, t=tstr) nested = False if self.isdependent(tkey): tsig = [ts for ts in self.refined_types if ts[0] == tkey][0] for ts, ti in zip(tsig[1:], tinst[1:]): if isinstance(ts, basestring): template_kw[ts] = self.cython_ctype(ti) else: template_kw[ti[0]] = ti[2] vartype = self.refined_types[tsig] if vartype in tsig[1:]: vartype = tinst[tsig.index(vartype)][1] if self.isrefinement(vartype): nested = True vdecl, vbody, vrtn = self.cython_py2c(var, vartype) template_kw['var'] = vrtn body_filled = body_template.format(**template_kw) if rtn_template: if '{t.cython_ctype}'in body_template: deft = tstr.cython_ctype elif '{t.cython_ctype_nopred}'in body_template: deft = tstr.cython_ctype_nopred elif '{t.cython_cytype_nopred}'in body_template: deft = tstr.cython_cytype_nopred else: deft = tstr.cython_cytype decl = "cdef {0} {1}".format(deft, proxy_name) body = body_filled rtn = rtn_template.format(**template_kw) decl += '\n'+"\n".join([l for l in body.splitlines() \ if l.startswith('cdef')]) body = "\n".join([l for l in body.splitlines() \ if not l.startswith('cdef')]) else: decl = body = None rtn = body_filled if nested: decl = '' if decl is None else decl vdecl = '' if vdecl is None else vdecl decl = (vdecl + '\n' + decl).strip() decl = None if 0 == len(decl) else decl body = '' if body is None else body vbody = '' if vbody is None else vbody body = (vbody + '\n' + body).strip() body = None if 0 == len(body) else body return decl, body, rtn ################# Some utility functions for the typesystem #############
[docs] def register_class(self, name=None, template_args=None, cython_c_type=None, cython_cimport=None, cython_cy_type=None, cython_py_type=None, cython_template_class_name=None, cython_template_function_name=None, cython_cyimport=None, cython_pyimport=None, cython_c2py=None, cython_py2c=None, cpp_type=None, human_name=None, from_pytype=None): """Classes are user specified types. This function will add a class to the type system so that it may be used normally with the rest of the type system. """ # register the class name isbase = True if template_args is None: self.base_types.add(name) # normal class elif isinstance(template_args, Sequence): if 0 == len(template_args): self.base_types.add(name) # normal class elif isinstance(template_args, basestring): _raise_type_error(name) else: self.template_types[name] = tuple(template_args) # templated class... isbase = False # Register with Cython C/C++ types if (cython_c_type is not None): self.cython_ctypes[name] = cython_c_type if (cython_cy_type is not None): self.cython_cytypes[name] = cython_cy_type if (cython_py_type is not None): self.cython_pytypes[name] = cython_py_type if (from_pytype is not None): self.from_pytypes[name] = from_pytype if cpp_type is not None: self.cpp_types[name] = cpp_type if human_name is not None: self.humannames[name] = human_name if (cython_cimport is not None): cython_cimport = _ensure_importable(cython_cimport) self.cython_cimports[name] = cython_cimport if (cython_cyimport is not None): cython_cyimport = _ensure_importable(cython_cyimport) self.cython_cyimports[name] = cython_cyimport if (cython_pyimport is not None): cython_pyimport = _ensure_importable(cython_pyimport) self.cython_pyimports[name] = cython_pyimport if (cython_c2py is not None): if isinstance(cython_c2py, basestring): cython_c2py = (cython_c2py,) cython_c2py = None if cython_c2py is None else tuple(cython_c2py) self.cython_c2py_conv[name] = cython_c2py if (cython_py2c is not None): if isinstance(cython_py2c, basestring): cython_py2c = (cython_py2c, False) self.cython_py2c_conv[name] = cython_py2c if (cython_template_class_name is not None): self.cython_classnames[name] = cython_template_class_name if (cython_template_function_name is not None): self.cython_functionnames[name] = cython_template_function_name
[docs] def deregister_class(self, name): """This function will remove a previously registered class from the type system. """ isbase = name in self.base_types if not isbase and name not in self.template_types: _raise_type_error(name) if isbase: self.base_types.remove(name) else: self.template_types.pop(name, None) self.cython_ctypes.pop(name, None) self.cython_cytypes.pop(name, None) self.cython_pytypes.pop(name, None) self.from_pytypes.pop(name, None) self.cpp_types.pop(name, None) self.humannames.pop(name, None) self.cython_cimports.pop(name, None) self.cython_cyimports.pop(name, None) self.cython_pyimports.pop(name, None) self.cython_c2py_conv.pop(name, None) self.cython_py2c_conv.pop(name, None) self.cython_classnames.pop(name, None) self.clearmemo()
[docs] def register_classname(self, classname, package, pxd_base, cpppxd_base, cpp_classname=None, make_dtypes=True): """Registers a class with the type system from only its name, and relevant header file information. Parameters ---------- classname : str or tuple package : str Package name where headers live. pxd_base : str Base name of the pxd file to cimport. cpppxd_base : str Base name of the cpppxd file to cimport. cpp_classname : str or tuple, optional Name of class in C++, equiv. to apiname.srcname. Defaults to classname. make_dtypes : bool, optional Flag for registering dtypes for this class simeltaneously with registering the class itself. """ # target classname baseclassname = classname if isinstance(classname, basestring): template_args = None templateclassname = baseclassname templatefuncname = baseclassname.lower() else: template_args = ['T{0}'.format(i) for i in range(len(classname)-2)] template_args = tuple(template_args) while not isinstance(baseclassname, basestring): baseclassname = baseclassname[0] # hack version of ts.basename() templateclassname = baseclassname templateclassname = templateclassname + \ ''.join(["{"+targ+"}" for targ in template_args]) templatefuncname = baseclassname.lower() + '_' + \ '_'.join(["{"+targ+"}" for targ in template_args]) # source classname if cpp_classname is None: cpp_classname = classname cpp_baseclassname = cpp_classname if not isinstance(cpp_classname, basestring): while not isinstance(cpp_baseclassname, basestring): cpp_baseclassname = cpp_baseclassname[0] if template_args is None: template_args = ['T{0}'.format(i) for i in range(len(cpp_classname)-2)] template_args = tuple(template_args) # register regular class class_c2py = ('{t.cython_pytype}({var})', ('{proxy_name} = {t.cython_pytype}()\n' '(<{t.cython_ctype_nopred} *> {proxy_name}._inst)[0] = {var}'), ('if {cache_name} is None:\n' ' {proxy_name} = {t.cython_pytype}()\n' ' {proxy_name}._free_inst = False\n' ' {proxy_name}._inst = &{var}\n' ' {cache_name} = {proxy_name}\n') ) class_py2c = ('{proxy_name} = <{t.cython_cytype_nopred}> {var}', '(<{t.cython_ctype_nopred} *> {proxy_name}._inst)[0]') class_cimport = ((package, cpppxd_base),) kwclass = dict( name=baseclassname, # FCComp template_args=template_args, cython_c_type=cpppxd_base + '.' + cpp_baseclassname, # cpp_fccomp.FCComp cython_cimport=class_cimport, cython_cy_type=pxd_base + '.' + baseclassname, # fccomp.FCComp cython_py_type=pxd_base + '.' + baseclassname, # fccomp.FCComp from_pytype=[pxd_base + '.' + baseclassname], # fccomp.FCComp cpp_type=cpp_baseclassname, human_name=templateclassname, cython_template_class_name=templateclassname, cython_template_function_name=templatefuncname, cython_cyimport=((pxd_base,),), # fccomp cython_pyimport=((pxd_base,),), # fccomp cython_c2py=class_c2py, cython_py2c=class_py2c, ) self.register_class(**kwclass) canonname = classname if isinstance(classname, basestring) \ else self.canon(classname) if template_args is not None: specname = classname if isinstance(classname, basestring) \ else self.cython_classname(classname)[1] kwclassspec = dict( name=classname, cython_c_type=cpppxd_base + '.' + specname, cython_cy_type=pxd_base + '.' + specname, cython_py_type=pxd_base + '.' + specname, cpp_type=self.cpp_type(cpp_classname), ) self.register_class(**kwclassspec) kwclassspec['name'] = canonname self.register_class(**kwclassspec) kwclassspec['name'] = self.canon(cpp_classname) self.register_class(**kwclassspec) # register numpy type if make_dtypes: self.register_numpy_dtype(classname, cython_cimport=class_cimport, cython_cyimport=pxd_base, cython_pyimport=pxd_base, ) # register vector class_vector_py2c = (( '# {var} is a {t.type}\n' 'cdef int i{var}\n' 'cdef int {var}_size\n' 'cdef {t.cython_npctypes[0]} * {var}_data\n' '{var}_size = len({var})\n' 'if isinstance({var}, np.ndarray) and (<np.ndarray> {var}).descr.type_num == {t.cython_nptype}:\n' ' {var}_data = <{t.cython_npctypes[0]} *> np.PyArray_DATA(<np.ndarray> {var})\n' ' {proxy_name} = {t.cython_ctype_nopred}(<size_t> {var}_size)\n' ' for i{var} in range({var}_size):\n' ' {proxy_name}[i{var}] = {var}_data[i{var}]\n' 'else:\n' ' {proxy_name} = {t.cython_ctype_nopred}(<size_t> {var}_size)\n' ' for i{var} in range({var}_size):\n' ' {proxy_name}[i{var}] = (<{t.cython_npctypes_nopred[0]} *> (<{t.cython_npcytypes_nopred[0]}> {var}[i{var}])._inst)[0]\n'), '{proxy_name}') self.register_class(('vector', canonname, 0), cython_py2c=class_vector_py2c) self.register_class(('vector', classname, 0), cython_py2c=class_vector_py2c) self.register_class((('vector', canonname, 0), '&'), cython_py2c=class_vector_py2c) self.register_class((('vector', classname, 0), '&'), cython_py2c=class_vector_py2c) self.register_class(((('vector', canonname, 0), 'const'), '&'), cython_py2c=class_vector_py2c) self.register_class(((('vector', classname, 0), 'const'), '&'), cython_py2c=class_vector_py2c) # register pointer to class class_ptr_c2py = ('{t.cython_pytype}({var})', ('cdef {t.cython_pytype} {proxy_name} = {t.cython_pytype}()\n' 'if {proxy_name}._free_inst:\n' ' free({proxy_name}._inst)\n' '(<{t.cython_ctype}> {proxy_name}._inst) = {var}'), ('if {cache_name} is None:\n' ' {proxy_name} = {t.cython_pytype}()\n' ' {proxy_name}._free_inst = False\n' ' {proxy_name}._inst = {var}\n' ' {cache_name} = {proxy_name}\n') ) class_ptr_py2c = ('{proxy_name} = <{t.cython_cytype_nopred}> {var}', '(<{t.cython_ctype_nopred} *> {proxy_name}._inst)') kwclassptr = dict( name=(classname, '*'), template_args=template_args, cython_py_type=pxd_base + '.' + baseclassname, cython_cy_type=pxd_base + '.' + baseclassname, cpp_type=cpp_baseclassname, cython_c2py=class_ptr_c2py, cython_py2c=class_ptr_py2c, cython_cimport=kwclass['cython_cimport'] , cython_cyimport=kwclass['cython_cyimport'] + (('libc.stdlib','free'),), cython_pyimport=kwclass['cython_pyimport'], ) self.register_class(**kwclassptr) kwclassref = dict(kwclassptr) # Register reference to class kwclassref['name'] = (classname, '&') kwclassref['cython_c2py'] = class_c2py kwclassref['cython_py2c'] = class_py2c #ts.register_class(**kwclassref) # register doublepointer to class class_dblptr_c2py = ('{t.cython_pytype}({var})', ('{proxy_name} = {proxy_name}_obj._inst\n' '(<{t.cython_ctype} *> {proxy_name}._inst) = {var}'), ('if {cache_name} is None:\n' ' {proxy_name} = {t.cython_pytype}()\n' ' {proxy_name}._free_inst = False\n' ' {proxy_name}._inst = {var}\n' ' {proxy_name}_list = [{proxy_name}]\n' ' {cache_name} = {proxy_name}_list\n') ) class_dblptr_py2c = ('{proxy_name} = <{t.cython_cytype_nopred}> {var}[0]', '(<{t.cython_ctype_nopred} **> {proxy_name}._inst)') kwclassdblptr = dict( name=((classname, '*'), '*'), template_args=template_args, cython_c2py=class_dblptr_c2py, cython_py2c=class_dblptr_py2c, cython_cimport=kwclass['cython_cimport'], cython_cyimport=kwclass['cython_cyimport'], cython_pyimport=kwclass['cython_pyimport'], ) self.register_class(**kwclassdblptr)
[docs] def register_refinement(self, name, refinementof, cython_cimport=None, cython_cyimport=None, cython_pyimport=None, cython_c2py=None, cython_py2c=None): """This function will add a refinement to the type system so that it may be used normally with the rest of the type system. """ self.refined_types[name] = refinementof cyci = _ensure_importable(cython_cimport) self.cython_cimports[name] = cyci cycyi = _ensure_importable(cython_cyimport) self.cython_cyimports[name] = cycyi cypyi = _ensure_importable(cython_pyimport) self.cython_pyimports[name] = cypyi if isinstance(cython_c2py, basestring): cython_c2py = (cython_c2py,) cython_c2py = None if cython_c2py is None else tuple(cython_c2py) if cython_c2py is not None: self.cython_c2py_conv[name] = cython_c2py if isinstance(cython_py2c, basestring): cython_py2c = (cython_py2c, False) if cython_py2c is not None: self.cython_py2c_conv[name] = cython_py2c
[docs] def deregister_refinement(self, name): """This function will remove a previously registered refinement from the type system. """ self.refined_types.pop(name, None) self.cython_c2py_conv.pop(name, None) self.cython_py2c_conv.pop(name, None) self.cython_cimports.pop(name, None) self.cython_cyimports.pop(name, None) self.cython_pyimports.pop(name, None) self.clearmemo()
[docs] def register_specialization(self, t, cython_c_type=None, cython_cy_type=None, cython_py_type=None, cython_cimport=None, cython_cyimport=None, cython_pyimport=None): """This function will add a template specialization so that it may be used normally with the rest of the type system. """ t = self.canon(t) if cython_c_type is not None: self.cython_ctypes[t] = cython_c_type if cython_cy_type is not None: self.cython_cytypes[t] = cython_cy_type if cython_py_type is not None: self.cython_pytypes[t] = cython_py_type if cython_cimport is not None: self.cython_cimports[t] = cython_cimport if cython_cyimport is not None: self.cython_cyimports[t] = cython_cyimport if cython_pyimport is not None: self.cython_pyimports[t] = cython_pyimport
[docs] def deregister_specialization(self, t): """This function will remove previously registered template specialization.""" t = self.canon(t) self.cython_ctypes.pop(t, None) self.cython_cytypes.pop(t, None) self.cython_pytypes.pop(t, None) self.cython_cimports.pop(t, None) self.cython_cyimports.pop(t, None) self.cython_pyimports.pop(t, None) self.clearmemo()
[docs] def register_numpy_dtype(self, t, cython_cimport=None, cython_cyimport=None, cython_pyimport=None): """This function will add a type to the system as numpy dtype that lives in the dtypes module. """ t = self.canon(t) if t in self.numpy_types: return varname = self.cython_variablename(t)[1] self.numpy_types[t] = '{dtypes}xd_' + varname + '.num' self.type_aliases[self.numpy_types[t]] = t self.type_aliases['xd_' + varname] = t self.type_aliases['xd_' + varname + '.num'] = t self.type_aliases['{dtypes}xd_' + varname] = t self.type_aliases['{dtypes}xd_' + varname + '.num'] = t if cython_cimport is not None: x = _ensure_importable(self.cython_cimports._d.get(t, None)) x = x + _ensure_importable(cython_cimport) self.cython_cimports[t] = x # cython imports x = (('{dtypes}',),) x = x + _ensure_importable(self.cython_cyimports._d.get(t, None)) x = x + _ensure_importable(cython_cyimport) self.cython_cyimports[t] = x # python imports x = (('{dtypes}',),) x = x + _ensure_importable(self.cython_pyimports._d.get(t, None)) x = x + _ensure_importable(cython_pyimport) self.cython_pyimports[t] = x
[docs] def register_argument_kinds(self, t, argkinds): """Registers an argument kind tuple into the type system for a template type. """ t = self.canon(t) if t in self.argument_kinds: old = self.argument_kinds[t] if old != argkinds: msg = ("overwriting argument kinds for type {0}:\n" " old: {1}\n" " new: {2}") warn(msg.format(t, old, argkinds), RuntimeWarning) self.argument_kinds[t] = argkinds
[docs] def deregister_argument_kinds(self, t): """Removes a type and its argument kind tuple from the type system.""" t = self.canon(t) if t in self.argument_kinds: del self.argument_kinds[t]
[docs] def register_variable_namespace(self, name, namespace, t=None): """Registers a variable and its namespace in the typesystem. """ if name in self.variable_namespace: old = self.variable_namespace[name] if old != namespace: msg = ("overwriting namespace for variable {0}:\n" " old: {1}\n" " new: {2}") warn(msg.format(name, old, namespace), RuntimeWarning) self.variable_namespace[name] = namespace if self.isenum(t): t = self.canon(t) for n, _ in t[1][2][2]: self.register_variable_namespace(n, namespace) #################### Type system helpers ###################################
[docs] def clearmemo(self): """Clears all method memoizations on this type system instance.""" # see utils.memozie_method if hasattr(self, '_cache'): self._cache.clear()
[docs] def delmemo(self, meth, *args, **kwargs): """Deletes a single key from a method on this type system instance.""" # see utils.memozie_method if hasattr(self, '_cache'): meth = getattr(self, meth )if isinstance(meth, basestring) else meth del self._cache[meth.func.meth, args, tuple(sorted(kwargs.items()))]
@contextmanager
[docs] def swap_dtypes(self, s): """A context manager for temporarily swapping out the dtypes value with a new value and replacing the original value before exiting.""" old = self.dtypes self.dtypes = s self.clearmemo() yield self.clearmemo() self.dtypes = old
@contextmanager
[docs] def swap_stlcontainers(self, s): """A context manager for temporarily swapping out the stlcontainer value with a new value and replacing the original value before exiting.""" old = self.stlcontainers self.stlcontainers = s self.clearmemo() yield self.clearmemo() self.stlcontainers = old
@contextmanager
[docs] def local_classes(self, classnames, typesets=frozenset(['cy', 'py'])): """A context manager for making sure the given classes are local.""" saved = {} for name in classnames: if 'c' in typesets and name in self.cython_ctypes: saved[name, 'c'] = _undot_class_name(name, self.cython_ctypes) if 'cy' in typesets and name in self.cython_cytypes: saved[name, 'cy'] = _undot_class_name(name, self.cython_cytypes) if 'py' in typesets and name in self.cython_pytypes: saved[name, 'py'] = _undot_class_name(name, self.cython_pytypes) self.clearmemo() yield for name in classnames: if 'c' in typesets and name in self.cython_ctypes: _redot_class_name(name, self.cython_ctypes, saved[name, 'c']) if 'cy' in typesets and name in self.cython_cytypes: _redot_class_name(name, self.cython_cytypes, saved[name, 'cy']) if 'py' in typesets and name in self.cython_pytypes: _redot_class_name(name, self.cython_pytypes, saved[name, 'py']) self.clearmemo() #################### Type System Above This Line ############################## ################### Type Matching #############################################o
[docs]class MatchAny(object): """A singleton helper class for matching any portion of a type.""" def __repr__(self): return "MatchAny" def __hash__(self): # give consistent hash value across executions return hash(repr(self))
MatchAny = MatchAny() """A singleton helper class for matching any portion of a type."""
[docs]class TypeMatcher(object): """A class that is used for checking whether a type matches a given pattern.""" def __init__(self, pattern): """Parameters ---------- pattern : nested tuples, str, int This is a type-like entity that may have certain elements replaced by MatchAny, indicating that any value in a concrete type may match this pattern. For example ('float64', MatchAny) will match the following:: ('float64', 0) ('float64', '*') ('float64', '&') ('float64', 'const') but will not match:: 'float64' (('float64', 'const'), '&') """ self._pattern = pattern @property
[docs] def pattern(self): """The pattern to match.""" # Make this field read-only to prevent hashing errors return self._pattern
def __hash__(self): # needed so that class can be dict key return hash(self.pattern)
[docs] def matches(self, t): """Tests that a type matches the pattern, returns True or False.""" pattern = self.pattern if pattern is MatchAny: return True if t is pattern: return True if pattern is None: return False if t == pattern: return True if isinstance(pattern, basestring): #return t == pattern if isinstance(t, basestring) else False return False if isinstance(t, basestring) or isinstance(t, bool) or \ isinstance(t, int) or isinstance(t, float): return False # now we know both pattern and t should be different non-string sequences, # nominally tuples or lists if len(t) != len(pattern): return False submatcher = TypeMatcher(None) for subt, subpattern in zip(t, pattern): submatcher._pattern = subpattern if not submatcher.matches(subt): return False return True
[docs] def flatmatches(self, t): """Flattens t and then sees if any part of it matches self.pattern. """ try: # See if user gave entire type if self.matches(t): return True except TypeError: # This might fail, if it does just move on pass else: if isinstance(t, basestring): return self.matches(t) elif isinstance(t, (tuple, list)): return any([self.matches(i) for i in flatten(t)])
def __eq__(self, other): if isinstance(other, TypeMatcher): return self._pattern == other._pattern else: return self._pattern == other def __str__(self): return "{0}({1!s})".format(self.__class__.__name__, self._pattern) def __repr__(self): return "{0}({1!r})".format(self.__class__.__name__, self._pattern)
[docs]def matches(pattern, t): """Indicates whether a type t matches a pattern. See TypeMatcher for more details. """ tm = TypeMatcher(pattern) return tm.matches(t) #################### Lazy Configuration ###################################
class _LazyConfigDict(MutableMapping): def __init__(self, items, ts): self._d = items if isinstance(items, MutableMapping) else dict(items) self._ts = ts def __len__(self): return len(self._d) def __contains__(self, key): return key in self._d def __iter__(self): for k in self._d: yield k def __getitem__(self, key): value = self._d[key] kw = {'extra_types': _ensuremoddot(self._ts.extra_types), 'dtypes': _ensuremoddot(self._ts.dtypes), 'stlcontainers': _ensuremoddot(self._ts.stlcontainers), } for k, v in kw.items(): value = _recurse_replace(value, '{' + k + '}', v) return value def __setitem__(self, key, value): self._d[key] = value def __delitem__(self, key): del self._d[key] def update(self, *args, **kwargs): if len(args) == 1 and len(kwargs) == 0: toup = args[0] if isinstance(toup, _LazyConfigDict): toup = toup._d elif len(args) == 0: toup = kwargs else: raise TypeError("invalid update signature.") self._d.update(toup) def __str__(self): return pformat(self._d) def __repr__(self): return self.__class__.__name__ + "(" + repr(self._d) + ", TypeSystem())" class _LazyImportDict(MutableMapping): def __init__(self, items, ts): self._d = items if isinstance(items, MutableMapping) else dict(items) self._ts = ts def __len__(self): return len(self._d) def __contains__(self, key): return key in self._d def __iter__(self): for k in self._d: yield k def __getitem__(self, key): value = self._d[key] if callable(value): return value kw = {'extra_types': _ensuremod(self._ts.extra_types), 'dtypes': _ensuremod(self._ts.dtypes), 'stlcontainers': _ensuremod(self._ts.stlcontainers),} newvalue = tuple(tuple(x.format(**kw) or None for x in imp if x is not None) \ for imp in value if imp is not None) or (None,) return newvalue def __setitem__(self, key, value): self._d[key] = value def __delitem__(self, key): del self._d[key] def update(self, *args, **kwargs): if len(args) == 1 and len(kwargs) == 0: toup = args[0] if isinstance(toup, _LazyImportDict): toup = toup._d elif len(args) == 0: toup = kwargs else: raise TypeError("invalid update signature.") self._d.update(toup) def __str__(self): return pformat(self._d) def __repr__(self): return self.__class__.__name__ + "(" + repr(self._d) + ", TypeSystem())" class _LazyConverterDict(MutableMapping): def __init__(self, items, ts): self._d = items if isinstance(items, MutableMapping) else dict(items) self._tms = set([k for k in self._d if isinstance(k, TypeMatcher)]) self._ts = ts def __len__(self): return len(self._d) def __contains__(self, key): if key in self._d: return True # Check if key is present else: # check if any TypeMatcher keys actually match for tm in self._tms: if tm.matches(key): self[key] = self._d[tm] return True else: return False def __iter__(self): for k in self._d: yield k def __getitem__(self, key): if key in self._d: value = self._d[key] # Check if key is present else: # check if any TypeMatcher keys actually match for tm in self._tms: if tm.matches(key): value = self._d[tm] self[key] = value break else: raise KeyError("{0} not found".format(key)) if value is None or value is NotImplemented or callable(value): return value kw = {'extra_types': _ensuremoddot(self._ts.extra_types), 'dtypes': _ensuremoddot(self._ts.dtypes), 'stlcontainers': _ensuremoddot(self._ts.stlcontainers),} newvalue = [] for x in value: newx = x if isinstance(newx, basestring): for k, v in kw.items(): newx = newx.replace('{' + k + '}', v) newvalue.append(newx) return tuple(newvalue) def __setitem__(self, key, value): self._d[key] = value if isinstance(key, TypeMatcher): self._tms.add(key) def __delitem__(self, key): del self._d[key] if isinstance(key, TypeMatcher): self._tms.remove(key) def update(self, *args, **kwargs): if len(args) == 1 and len(kwargs) == 0: toup = args[0] if isinstance(toup, _LazyConverterDict): toup = toup._d elif len(args) == 0: toup = kwargs else: raise TypeError("invalid update signature.") self._d.update(toup) self._tms.update([k for k in toup if isinstance(k, TypeMatcher)]) def __str__(self): return pformat(self._d) def __repr__(self): return self.__class__.__name__ + "(" + repr(self._d) + ", TypeSystem())" #################### Type string for formatting ################################
[docs]class typestr(object): """This is class whose attributes are properties that expose various string representations of a type. This is useful for the Python string formatting mini-language where attributes of an object may be accessed. For example: "This is the Cython C/C++ type: {t.cython_ctype}".format(t=typestr(t, ts)) This mechanism is used for accessing type information in conversion strings. """ def __init__(self, t, ts): """Parameters ---------- t : str or tuple A valid repesentation of a type in the type systems ts : TypeSystem A type system to generate the string representations with. """ self.ts = ts self.t = ts.canon(t) self.t_nopred = ts.strip_predicates(t) _type = None @property
[docs] def type(self): """This is a repr string of the raw type (self.t), mostly useful for comments.""" if self._type is None: self._type = repr(self.t) return self._type
_cython_ctype = None @property
[docs] def cython_ctype(self): """The Cython C/C++ representation of the type. """ if self._cython_ctype is None: self._cython_ctype = self.ts.cython_ctype(self.t) return self._cython_ctype
_cython_cytype = None @property
[docs] def cython_cytype(self): """The Cython Cython representation of the type. """ if self._cython_cytype is None: self._cython_cytype = self.ts.cython_cytype(self.t) return self._cython_cytype
_cython_pytype = None @property
[docs] def cython_pytype(self): """The Cython Python representation of the type. """ if self._cython_pytype is None: self._cython_pytype = self.ts.cython_pytype(self.t) return self._cython_pytype
_cython_nptype = None @property
[docs] def cython_nptype(self): """The Cython NumPy representation of the type. """ if self._cython_nptype is None: self._cython_nptype = self.ts.cython_nptype(self.t) return self._cython_nptype
_cython_npctype = None @property
[docs] def cython_npctype(self): """The Cython C/C++ representation of NumPy type. """ if self._cython_npctype is None: npt = self.ts.cython_nptype(self.t) npct = self.ts.cython_ctype(npt) self._cython_npctype = npct return self._cython_npctype
_cython_npcytype = None @property
[docs] def cython_npcytype(self): """The Cython Cython representation of NumPy type. """ if self._cython_npcytype is None: npt = self.ts.cython_nptype(self.t) npcyt = self.ts.cython_cytype(npt) self._cython_npcytype = npcyt return self._cython_npcytype
_cython_nppytype = None @property
[docs] def cython_nppytype(self): """The Cython Python representation of NumPy type. """ if self._cython_nppytype is None: npt = self.ts.cython_nptype(self.t) nppyt = self.ts.cython_pytype(npt) self._cython_nppytype = nppyt return self._cython_nppytype
_cython_nptypes = None @property
[docs] def cython_nptypes(self): """The expanded Cython NumPy representation of the type. """ if self._cython_nptypes is None: npts = self.ts.cython_nptype(self.t, depth=1) npts = [npts] if isinstance(npts, basestring) else npts self._cython_nptypes = npts return self._cython_nptypes
_cython_npctypes = None @property
[docs] def cython_npctypes(self): """The expanded Cython C/C++ representation of the NumPy types. """ if self._cython_npctypes is None: npts = self.ts.cython_nptype(self.t, depth=1) npts = [npts] if isinstance(npts, basestring) else npts npcts = _maprecurse(self.ts.cython_ctype, npts) self._cython_npctypes = npcts return self._cython_npctypes
_cython_npcytypes = None @property
[docs] def cython_npcytypes(self): """The expanded Cython Cython representation of the NumPy types. """ if self._cython_npcytypes is None: npts = self.ts.cython_nptype(self.t, depth=1) npts = [npts] if isinstance(npts, basestring) else npts npcyts = _maprecurse(self.ts.cython_cytype, npts) self._cython_npcytypes = npcyts return self._cython_npcytypes
_cython_nppytypes = None @property
[docs] def cython_nppytypes(self): """The expanded Cython Cython representation of the NumPy types. """ if self._cython_nppytypes is None: npts = self.ts.cython_nptype(self.t, depth=1) npts = [npts] if isinstance(npts, basestring) else npts nppyts = _maprecurse(self.ts.cython_pytype, npts) self._cython_nppytypes = nppyts return self._cython_nppytypes
_type_nopred = None @property
[docs] def type_nopred(self): """This is a repr string of the raw type (self.t) without predicates.""" if self._type_nopred is None: self._type_nopred = repr(self.t_nopred) return self._type_nopred
_cython_ctype_nopred = None @property
[docs] def cython_ctype_nopred(self): """The Cython C/C++ representation of the type without predicates. """ if self._cython_ctype_nopred is None: self._cython_ctype_nopred = self.ts.cython_ctype(self.t_nopred) return self._cython_ctype_nopred
_cython_cytype_nopred = None @property
[docs] def cython_cytype_nopred(self): """The Cython Cython representation of the type without predicates. """ if self._cython_cytype_nopred is None: self._cython_cytype_nopred = self.ts.cython_cytype(self.t_nopred) return self._cython_cytype_nopred
_cython_pytype_nopred = None @property
[docs] def cython_pytype_nopred(self): """The Cython Python representation of the type without predicates. """ if self._cython_pytype_nopred is None: self._cython_pytype_nopred = self.ts.cython_pytype(self.t_nopred) return self._cython_pytype_nopred
_cython_nptype_nopred = None @property
[docs] def cython_nptype_nopred(self): """The Cython NumPy representation of the type without predicates. """ if self._cython_nptype_nopred is None: self._cython_nptype_nopred = self.ts.cython_nptype(self.t_nopred) return self._cython_nptype_nopred
_cython_npctype_nopred = None @property
[docs] def cython_npctype_nopred(self): """The Cython C/C++ representation of the NumPy type without predicates. """ if self._cython_npctype_nopred is None: npt_nopred = self.ts.cython_nptype(self.t_nopred) npct_nopred = self.cython_ctype(npt_nopred) self._cython_npctype_nopred = npct_nopred return self._cython_npctype_nopred
_cython_npcytype_nopred = None @property
[docs] def cython_npcytype_nopred(self): """The Cython Cython representation of the NumPy type without predicates. """ if self._cython_npcytype_nopred is None: npt_nopred = self.ts.cython_nptype(self.t_nopred) npcyt_nopred = self.cython_cytype(npt_nopred) self._cython_npcytype_nopred = npcyt_nopred return self._cython_npcytype_nopred
_cython_nppytype_nopred = None @property
[docs] def cython_nppytype_nopred(self): """The Cython Python representation of the NumPy type without predicates. """ if self._cython_nppytype_nopred is None: npt_nopred = self.ts.cython_nptype(self.t_nopred) nppyt_nopred = self.cython_pytype(npt_nopred) self._cython_nppytype_nopred = nppyt_nopred return self._cython_nppytype_nopred
_cython_nptypes_nopred = None @property
[docs] def cython_nptypes_nopred(self): """The Cython NumPy representation of the types without predicates. """ if self._cython_nptypes_nopred is None: self._cython_nptypes_nopred = self.ts.cython_nptype(self.t_nopred, depth=1) return self._cython_nptypes_nopred
_cython_npctypes_nopred = None @property
[docs] def cython_npctypes_nopred(self): """The Cython C/C++ representation of the NumPy types without predicates. """ if self._cython_npctypes_nopred is None: npts_nopred = self.ts.cython_nptype(self.t_nopred, depth=1) npts_nopred = [npts_nopred] if isinstance(npts_nopred, basestring) \ else npts_nopred npcts_nopred = _maprecurse(self.ts.cython_ctype, npts_nopred) self._cython_npctypes_nopred = npcts_nopred return self._cython_npctypes_nopred
_cython_npcytypes_nopred = None @property
[docs] def cython_npcytypes_nopred(self): """The Cython Cython representation of the NumPy types without predicates. """ if self._cython_npcytypes_nopred is None: npts_nopred = self.ts.cython_nptype(self.t_nopred, depth=1) npts_nopred = [npts_nopred] if isinstance(npts_nopred, basestring) \ else npts_nopred npcyts_nopred = _maprecurse(self.ts.cython_cytype, npts_nopred) self._cython_npcytypes_nopred = npcyts_nopred return self._cython_npcytypes_nopred
_cython_nppytypes_nopred = None @property
[docs] def cython_nppytypes_nopred(self): """The Cython Python representation of the NumPy types without predicates. """ if self._cython_nppytypes_nopred is None: npts_nopred = self.ts.cython_nptype(self.t_nopred, depth=1) npts_nopred = [npts_nopred] if isinstance(npts_nopred, basestring) \ else npts_nopred nppyts_nopred = _maprecurse(self.ts.cython_pytype, npts_nopred) self._cython_nppytypes_nopred = nppyts_nopred return self._cython_nppytypes_nopred #################### Type system helpers #######################################
def _raise_type_error(t): raise TypeError("type of {0!r} could not be determined".format(t)) _ensuremod = lambda x: x if x is not None and 0 < len(x) else '' _ensuremoddot = lambda x: x + '.' if x is not None and 0 < len(x) else '' def _recurse_replace(x, a, b): if isinstance(x, basestring): return x.replace(a, b) elif isinstance(x, Sequence): return tuple([_recurse_replace(y, a, b) for y in x]) else: return x def _undot_class_name(name, d): value = d[name] if '.' not in value: return '' v1, v2 = value.rsplit('.', 1) d[name] = v2 return v1 def _redot_class_name(name, d, value): if 0 == len(value): return d[name] = value + '.' + d[name] def _maprecurse(f, x): if not isinstance(x, list): return [f(x)] #return [_maprecurse(f, y) for y in x] l = [] for y in x: l += _maprecurse(f, y) return l def _ensure_importable(x): if isinstance(x, basestring) or x is None: r = ((x,),) elif isinstance(x, Iterable) and (isinstance(x[0], basestring) or x[0] is None): r = (x,) else: r = x return r