API Reference¶
This page contains the complete API reference for the mappingtools library.
Aggregation¶
Aggregation
¶
Bases: Enum
ALL = Item(collection_type=list, func=all_aggregator)
class-attribute
instance-attribute
¶
Aggregate all values into a list.
COUNT = Item(collection_type=Counter, func=count_aggregator)
class-attribute
instance-attribute
¶
Count occurrences of each value.
DISTINCT = Item(collection_type=set, func=distinct_aggregator)
class-attribute
instance-attribute
¶
Aggregate distinct values into a set.
EMA = Item(collection_type=float, func=ema_aggregator)
class-attribute
instance-attribute
¶
Calculate the exponential moving average of values.
FIRST = Item(collection_type=None, func=first_aggregator)
class-attribute
instance-attribute
¶
Take the first value encountered.
LAST = Item(collection_type=None, func=last_aggregator)
class-attribute
instance-attribute
¶
Take the last value encountered.
MAX = Item(collection_type=float, func=max_aggregator)
class-attribute
instance-attribute
¶
Take the maximum value.
MIN = Item(collection_type=float, func=min_aggregator)
class-attribute
instance-attribute
¶
Take the minimum value.
SUM = Item(collection_type=float, func=sum_aggregator)
class-attribute
instance-attribute
¶
Sum all values.
aggregator
property
¶
Return the aggregator function for this mode.
collection_type
property
¶
Return the collection type used for this aggregation mode.
all_aggregator(mapping, key, values)
¶
Extends the list at mapping[key] with values.
count_aggregator(mapping, key, values)
¶
Updates the Counter at mapping[key] with values.
distinct_aggregator(mapping, key, values)
¶
Updates the set at mapping[key] with values.
ema_aggregator(mapping, key, values)
¶
Updates mapping[key] to the exponential moving average (alpha=0.5) of its current value and the values.
first_aggregator(mapping, key, values)
¶
Sets mapping[key] to the first value if the key does not exist.
last_aggregator(mapping, key, values)
¶
Sets mapping[key] to the last value in the iterable.
max_aggregator(mapping, key, values)
¶
Updates mapping[key] to the maximum of its current value and the max of values.
min_aggregator(mapping, key, values)
¶
Updates mapping[key] to the minimum of its current value and the min of values.
sum_aggregator(mapping, key, values)
¶
Adds the sum of values to mapping[key].
Collectors¶
AutoMapper
¶
Bases: Mapping
A Mapping that automatically generates and assigns unique, minified strings for any new keys accessed. The minified keys are generated using the specified alphabet.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
alphabet
|
str
|
str - The alphabet to use for generating minified keys. Default is uppercase ASCII letters (A-Z). |
ascii_uppercase
|
Example
CategoryCollector
¶
Bases: defaultdict[str, MappingCollector[Category, VT_co]]
CategoryCollector is a specialized collector that organizes values into categories based on specified criteria.
It extends defaultdict to automatically create MappingCollector instances for each category as needed.
The collector supports both constant and dynamic categories, allowing for flexible data organization and
aggregation based on the specified mode.
__init__(aggregation=Aggregation.ALL, **kwargs)
¶
Initialize the CategoryCollector with the specified aggregation mode.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
aggregation
|
Aggregation
|
The mode for collecting mappings. |
ALL
|
add(data, **categories)
¶
Add a single value to the appropriate category in the collector.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
VT_co
|
The value to be added to the mapping. |
required |
**categories
|
Category | Callable[[VT_co], Category]
|
Keyword arguments where keys are category names and values are either category values or callables that return category values. |
{}
|
Returns:
| Type | Description |
|---|---|
|
None |
collect(iterable, **categories)
¶
Collect values from the given iterable and add them to the appropriate categories in the collector.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
iterable
|
Iterable[VT_co]
|
An iterable containing values to collect. |
required |
**categories
|
Category | Callable[[Iterable[VT_co]], Category]
|
Keyword arguments where keys are category names and values are either category values or callables that return category values. |
{}
|
Returns:
| Type | Description |
|---|---|
|
None |
CategoryCounter
¶
Bases: CategoryCollector
CategoryCounter is a specialized version of CategoryCollector that counts occurrences of values within
categories.
__init__(**kwargs)
¶
Initialize the CategoryCounter.
add(data, **categories)
¶
Add a single value to the appropriate category in the collector.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
VT_co
|
The value to be added to the mapping. |
required |
**categories
|
Category | Callable[[VT_co], Category]
|
Keyword arguments where keys are category names and values are either category values or callables that return category values. |
{}
|
Returns:
| Type | Description |
|---|---|
|
None |
collect(iterable, **categories)
¶
Collect values from the given iterable and add them to the appropriate categories in the collector.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
iterable
|
Iterable[VT_co]
|
An iterable containing values to collect. |
required |
**categories
|
Category | Callable[[Iterable[VT_co]], Category]
|
Keyword arguments where keys are category names and values are either category values or callables that return category values. |
{}
|
Returns:
| Type | Description |
|---|---|
|
None |
MappingCollector
¶
Bases: Generic[KT, VT_co]
MappingCollector is a flexible utility for collecting key-value pairs based on a specified aggregation mode.
It supports various modes of aggregation, such as collecting all values for a key, counting occurrences,
or applying custom aggregation functions.
The class is initialized based on the collection type associated with the chosen aggregation mode,
allowing for efficient storage and retrieval of collected data.
Public Methods
add(key: KT, *values: VT): Add one or more values to the internal mapping based on the specified mode.collect(iterable: Iterable[tuple[KT, VT]]): Collect key-value pairs from the given iterable based on the specified mode.
mapping
property
¶
Return a shallow copy of the internal mapping.
Returns:
| Type | Description |
|---|---|
dict[KT, VT_co]
|
dict[KT, VT_co]: A shallow copy of the internal mapping. |
__init__(aggregation=Aggregation.ALL, **kwargs)
¶
Initialize the MappingCollector with the specified mode.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
aggregation
|
Aggregation
|
The mode for collecting mappings. |
ALL
|
**kwargs
|
Variable keyword arguments used to initialize the internal mapping. |
{}
|
add(key, *values)
¶
Add one or more values to the internal mapping based on the specified mode.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
KT
|
The key to be added to the mapping. |
required |
*values
|
VT
|
The values corresponding to the key. |
()
|
Returns:
| Type | Description |
|---|---|
|
None |
MeteredDict
¶
Bases: dict[KT, VT_co]
A dictionary that tracks access and modification statistics for its keys.
operations
cached
property
¶
Returns the list of active operations being tracked.
count(key, operations=None)
¶
Returns the number of times a key has been accessed.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
KT
|
The key to check access count for. |
required |
operations
|
DictOperation
|
The operation of tracking to check. |
None
|
counts(operations=None)
¶
frequencies(operations=None)
¶
frequency(key, operations=None)
¶
Returns the frequency of access for a key.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
KT
|
The key to check access frequency for. |
required |
operations
|
DictOperation | None
|
The operation to check. If None, checks all operations. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
float |
dict[str, float]
|
The frequency of access for the key. |
get(key, default=None)
¶
Returns the value for the specified key if it exists, otherwise returns the default value. If the key does not exist, it increments the default access count.
reset(key=None, operations=None)
¶
Resets the tracking information for the specified operation and key.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
KT | None
|
The key to reset tracking information for (default is None, which resets all keys). |
None
|
operations
|
DictOperation | None
|
The operations to reset (default is None, which resets all). |
None
|
Returns:
| Type | Description |
|---|---|
|
None |
setdefault(key, default=None)
¶
Returns the value for the specified key if it exists, otherwise sets it to the default value. If the key does not exist, it increments the default access count.
summaries(operations=None)
¶
summary(key, operations=None)
¶
unused_keys(operations=None)
¶
Returns a list of keys that have never been accessed.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
operations
|
DictOperation | None
|
The operation to check. If None, checks all operations. |
None
|
Returns:
| Type | Description |
|---|---|
list[KT]
|
list[KT]: A list of keys that have never been accessed. |
used_keys(min_count=0, max_count=float('inf'), min_frequency=0.0, max_frequency=float('inf'), before=None, after=None, operations=None)
¶
Returns a list of keys that have been accessed at least once.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
min_count
|
int
|
Minimum number of accesses for the key to be included (default is 0). |
0
|
max_count
|
int
|
Maximum number of accesses for the key to be included (default is float('inf')). |
float('inf')
|
min_frequency
|
float
|
Minimum frequency of access for the key to be included (default is 0.0). |
0.0
|
max_frequency
|
float
|
Maximum frequency of access for the key to be included (default is float('inf')). |
float('inf')
|
before
|
datetime | None
|
If specified, only keys accessed before this datetime will be included (default is None). |
None
|
after
|
datetime | None
|
If specified, only keys accessed after this datetime will be included (default is None). |
None
|
operations
|
DictOperation | None
|
The operation to check. If None, checks all operations. |
None
|
Returns:
| Type | Description |
|---|---|
list[KT]
|
list[KT]: A list of keys that have been accessed. |
nested_defaultdict(nesting_depth=0, default_factory=None, **kwargs)
¶
Return a nested defaultdict with the specified nesting depth and default factory. A nested_defaultdict with nesting_depth=0 is equivalent to builtin 'collections.defaultdict'. For each increment in nesting_depth an additional item accessor is added.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
nesting_depth
|
int
|
The depth of nesting for the defaultdict (default is 0); |
0
|
default_factory
|
Callable
|
The default factory function for the defaultdict (default is None). |
None
|
**kwargs
|
Additional keyword arguments to initialize the most nested defaultdict. |
{}
|
Returns:
| Name | Type | Description |
|---|---|---|
defaultdict |
defaultdict
|
A nested defaultdict based on the specified parameters. |
Operators¶
combine(tree1=MISSING, tree2=MISSING, op=Resolver.LAST)
¶
Combines two trees using a binary operator op that resolves conflicts at the leaf nodes.
This is a powerful generalization of merge. It recursively walks two tree
structures and applies the op only when it encounters a conflict at the
leaf nodes (e.g., two scalars at the same path, or a structural mismatch).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
tree1
|
Tree[T] | Missing
|
The first tree structure. |
MISSING
|
tree2
|
Tree[T] | Missing
|
The second tree structure. |
MISSING
|
op
|
Combine | ResolverType
|
A callable or a Resolver enum strategy (Resolver, LogicalResolver, NumericResolver)
to resolve collisions. Defaults to |
LAST
|
Returns:
| Type | Description |
|---|---|
Tree[T] | Any
|
Tree[T] | Any: The combined tree structure. |
distinct(key, *mappings)
¶
Yield distinct values for the specified key across multiple mappings.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
K
|
The key to extract distinct values from the mappings. |
required |
*mappings
|
Mapping[K, Any]
|
Variable number of mappings to search for distinct values. |
()
|
Yields:
| Type | Description |
|---|---|
Any
|
Generator[K, Any, None]: A generator of distinct values extracted from the mappings. |
flatten(mapping, delimiter=None)
¶
Flatten a nested mapping structure into a single-level dictionary.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
mapping
|
Mapping[Any, Any]
|
The nested mapping to flatten. |
required |
delimiter
|
str | None
|
Uses this delimiter to join the path parts. If None then return path tuple. |
None
|
Returns:
| Type | Description |
|---|---|
dict[tuple | str, Any]
|
dict |
inverse(mapping)
¶
merge(tree1=MISSING, tree2=MISSING)
¶
A pure function (Monoid operation) to deeply merge two recursive tree structures.
The merging strategy resolves conflicts by overwriting existing values with new ones (right-side precedence).
MISSING acts as the identity element.
Mathematically, this operation forms a composite Monoid:
- Last Monoid (Scalar Fallback): When resolving conflicts between simple values, the right-hand
side (tree2) wins.
- Pointwise Monoid (Dictionary Merge): If the values are dictionaries, they are merged by key,
recursively calling merge on the values.
- Zip Monoid (List Merge): If both are lists, they are zipped and merged positionally,
substituting MISSING for missing indices.
- Free Monoid (Mixed List/Scalar): If one is a list and the other is a scalar/dict,
it concatenates (appends/prepends).
Because it forms a Monoid, this function can be used with functools.reduce to collect
an iterable of trees into a single structure.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
tree1
|
Tree[T] | Missing
|
The first tree structure. |
MISSING
|
tree2
|
Tree[T] | Missing
|
The second tree structure. |
MISSING
|
Returns:
| Type | Description |
|---|---|
Tree[T]
|
Tree[T] | Missing: The deeply merged tree structure. |
pivot(iterable, *, index, columns, values, aggregation=Aggregation.LAST)
¶
Reshape data (produce a "pivot" table) based on column values.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
iterable
|
Iterable[Mapping]
|
An iterable of mappings (e.g., list of dicts). |
required |
index
|
str
|
The key to use for the row labels. |
required |
columns
|
str
|
The key to use for the column labels. |
required |
values
|
str
|
The key to use for the values. |
required |
aggregation
|
Aggregation
|
The aggregation mode to use for values. Defaults to Aggregation.LAST. |
LAST
|
Returns:
| Type | Description |
|---|---|
dict[Any, dict[Any, Any]]
|
A nested dictionary: {index_value: {column_value: aggregated_value}}. |
rekey(mapping, key_factory, *, aggregation=Aggregation.LAST)
¶
Transform keys of a mapping based on a factory function of (key, value).
This allows "re-indexing" a mapping where the new key depends on the content of the value or a combination of the old key and value. Collisions are handled according to the specified aggregation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
mapping
|
Mapping[Any, Any]
|
The source mapping. |
required |
key_factory
|
Callable[[Any, Any], K]
|
A callable that takes (key, value) and returns the new key. |
required |
aggregation
|
Aggregation
|
How to handle key collisions. Defaults to Aggregation.LAST. |
LAST
|
Returns:
| Type | Description |
|---|---|
dict[K, Any]
|
A new dictionary with keys generated by the factory and aggregated values. |
rename(mapping, mapper, *, aggregation=Aggregation.LAST)
¶
Rename keys in a mapping based on a mapper (Mapping or Callable).
This operator creates a new dictionary with renamed keys. If a key is not present in the mapper, it remains unchanged. Collisions (when multiple original keys map to the same new key) are handled according to the specified aggregation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
mapping
|
Mapping[K, Any]
|
The source mapping. |
required |
mapper
|
Mapping[K, K] | Callable[[K], K]
|
A dictionary mapping old keys to new keys, or a function that transforms keys. |
required |
aggregation
|
Aggregation
|
How to handle key collisions. Defaults to Aggregation.LAST. |
LAST
|
Returns:
| Type | Description |
|---|---|
dict[K, Any]
|
A new dictionary with renamed keys and aggregated values. |
reshape(iterable, keys, value, aggregation=Aggregation.LAST)
¶
Reshape a stream of mappings into a nested dictionary (tensor) of arbitrary depth.
This is a generalization of pivot that supports N-dimensional nesting.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
iterable
|
Iterable[Mapping]
|
An iterable of mappings (records). |
required |
keys
|
Sequence[str | Callable[[Mapping], Any]]
|
A sequence of keys (or callables) to use for the nesting hierarchy. |
required |
value
|
str | Callable[[Mapping], Any]
|
The key (or callable) to use for the leaf values. |
required |
aggregation
|
Aggregation
|
The aggregation mode to use for collisions at the leaf. |
LAST
|
Returns:
| Type | Description |
|---|---|
dict[Any, Any]
|
A nested dictionary where the depth equals len(keys). |
Optics¶
Lens
¶
Bases: Generic[T, U]
A functional optic that focuses on a specific part of a data structure.
Lenses allow you to get, set, and modify deeply nested data in an immutable way.
They are composable using the / operator, similar to pathlib.
Example
>>> data = {"user": {"profile": {"name": "Alice"}}}
>>> # Path-like composition with auto-inference for keys/indices
>>> name_lens = Lens.key("user") / "profile" / "name"
>>> name_lens.get(data)
'Alice'
>>> # Lenses are callable (alias for get)
>>> name_lens(data)
'Alice'
>>> new_data = name_lens.set(data, "Bob")
>>> new_data["user"]["profile"]["name"]
'Bob'
>>> data["user"]["profile"]["name"] # Original is unchanged
'Alice'
__call__(source)
¶
Alias for get(). Allows the lens to be used as a function.
__rtruediv__(other)
¶
Allows composition when the left operand is not a Lens. Example: "users" / Lens.key("name")
__truediv__(other)
¶
Composes this lens with another lens using the / operator. If 'other' is not a Lens, it is treated as a key/index item.
attr(name)
staticmethod
¶
Creates a lens that focuses on an object attribute.
get(source)
¶
Extracts the focus value from the source.
index(i)
staticmethod
¶
Creates a lens that focuses on a list index.
item(k)
staticmethod
¶
Creates a smart lens that focuses on an item (key or index). It detects the container type at runtime to ensure correct immutable setting.
key(k)
staticmethod
¶
Creates a lens that focuses on a dictionary key.
modify(source, func)
¶
Modifies the focus value using a function.
path(*segments)
staticmethod
¶
Creates a lens from a sequence of keys/indices. Example: Lens.path("users", 0, "name")
set(source, value)
¶
Sets the focus value, returning a new source object (if supported).
patch(data, changes)
¶
Applies a set of changes to a data structure immutably.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
T
|
The source data structure. |
required |
changes
|
Mapping[str | Lens, Any]
|
A mapping of paths (dot-separated strings or Lenses) to new values. |
required |
Returns:
| Type | Description |
|---|---|
T
|
A new data structure with the changes applied. |
project(data, schema)
¶
Projects a data structure into a new dictionary shape based on a schema.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
Any
|
The source data structure. |
required |
schema
|
Mapping[str, str | Lens]
|
A mapping of output keys to source paths (dot-separated strings or Lenses). |
required |
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
A new dictionary containing the projected values. |
Example
data = {"user": {"name": "Alice", "version": 1}} project(data, {"username": "user.name", "v": "user.version"})
Structures¶
Dictifier
¶
Bases: dict[str, T], Generic[T]
A dict-like object that proxies attribute access to its values.
This class requires an explicit type to be provided, either through generic type hinting (e.g., Dictifier[MyClass]) or by using the @dictify decorator. It will not infer the type from its contents.
Method Chaining and Type Safety
When you call a proxied method, it attempts to determine the return
type from the method's type hints. If successful, it returns a new
strict Dictifier. If not, it falls back to an AutoDictifier
to allow chaining via type inference.
For a version that always infers types, see AutoDictifier.
__getattr__(name)
¶
Proxies attribute access to the contained objects.
auto(source)
classmethod
¶
Creates a Dictifier that automatically infers types from its contents.
of(target_class)
classmethod
¶
Creates a specialized Dictifier subclass for the given target class.
This method inspects the target class and pre-compiles proxy methods for all its public methods and properties. This provides significantly better performance than the standard dynamic Dictifier.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
target_class
|
type[T]
|
The class to wrap. |
required |
Returns:
| Type | Description |
|---|---|
type[Dictifier[T]]
|
A new class that inherits from Dictifier and has specialized proxies. |
LazyDictifier
¶
Bases: Mapping[str, T], Generic[T]
A dict-like object that lazily proxies attribute access to its values.
Operations (method calls, attribute access) are not executed immediately. Instead, they build up a pipeline of operations. The pipeline is only executed for a specific key when that key is accessed.
__call__(*args, **kwargs)
¶
Adds a method call operation to the chain.
__getattr__(name)
¶
Adds a new operation to the chain.
__getitem__(key)
¶
Executes the operation chain for a specific key.
__iter__()
¶
Iterates over the keys of the original source.
__len__()
¶
Returns the length of the original source.
map_objects(source, *, lazy=False, type_hint=None)
¶
Creates a proxy mapping for a collection of objects.
This factory function provides a unified entry point for creating Dictifier or LazyDictifier instances based on the desired behavior.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source
|
Mapping[str, T]
|
The source mapping of objects (e.g., a dict). |
required |
lazy
|
bool
|
If True, returns a LazyDictifier that defers execution. If False (default), returns an eager Dictifier. |
False
|
type_hint
|
type[T] | None
|
Optional class type of the objects in the source. If provided, returns a strict Dictifier. If None, returns a Dictifier in auto-inference mode. |
None
|
Returns:
| Type | Description |
|---|---|
Mapping[str, T]
|
A Mapping that proxies attribute access to the contained objects. |
Transformers¶
Transformer
¶
A class to transform objects recursively based on their type.
__call__(obj)
¶
__init__(mapping_handler=None, iterable_handler=None, class_handler=None, default_handler=None, *args, **kwargs)
¶
Initialize the Transformer with optional handlers for different types of objects.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
mapping_handler
|
Optional[Callable]
|
Handler for mapping objects. |
None
|
Iterable_handler
|
Optional[Callable]
|
Handler for iterable objects. |
required |
Class_handler
|
Optional[Callable]
|
Handler for class instances. |
required |
Default_handler
|
Optional[Callable]
|
Default handler for other objects. |
required |
*args
|
Additional positional arguments for handlers. |
()
|
|
**kwargs
|
Additional keyword arguments for handlers. |
{}
|
Typing¶
EnhancedJsonTree = T | JsonTree | list['EnhancedJsonTree[T]'] | dict[str, 'EnhancedJsonTree[T]']
module-attribute
¶
EnhancedJsonTree is a recursive type that extends JsonTree by allowing each node to also be of type T, in addition to JSON scalars, lists of enhanced JSON trees, or dictionaries mapping strings to enhanced JSON trees.
JsonScalar = None | bool | int | float | str
module-attribute
¶
JsonScalar represents the basic scalar types found in JSON data.
JsonTree = Tree[JsonScalar]
module-attribute
¶
JsonTree is a recursive type representing a JSON-like tree structure where each node can be a JSON scalar, a list of JSON trees, or a dictionary mapping strings to JSON trees.
MISSING = object()
module-attribute
¶
Sentinel object used to distinguish an explicit missing value from an actual None value.
Used by the operators.merge function
Missing = TypeVar('Missing', bound=(type(MISSING)))
module-attribute
¶
Type variable representing the type of the MISSING sentinel object. This allows for type checking to recognize the MISSING object as a distinct type.
Tree = T | list['Tree[T]'] | dict[str, 'Tree[T]']
module-attribute
¶
Tree is a recursive type representing a tree structure where each node can be of type T, a list of subtrees, or a dictionary mapping strings to subtrees.