KeyedList¶

KeyedList ¶

A row that reads like a dict but points at a list instead of copying its values.

A KeyedList pairs a list of values with an index of the keys. You read it by key, as in row['qty'], the same way you read a dict. But the values are not copied into it. It adopts the list you give it. It can also share one index of keys with many other KeyedLists, which is what a table does with the column names of its rows.

Daffodil gives you its rows as KeyedLists when itermode is 'keyedlist', and from iter_klist() and iloc(). Each one points at a row of the table.

How it differs from a dict:

Writing to it writes to the list it adopted. If that list is a row of a Daf, the Daf changes. A dict row never changes the table.
values() returns the adopted list itself, not a copy. Use to_dict() for an independent copy.
Creating one is cheap if you give it an existing index. A dict copies every value in.
Keys must be unique, and keys and values must be the same length.

Ways to create one:

KeyedList(keys, values): adopts the values list. No copy.
KeyedList(hd, values): the same, but shares an hd that you already have, a dict of each key and its position. This is the fastest, and it is what Daf does for its rows: each row shares the hd of the Daf. Then row.hd is daf.hd.
KeyedList(a_dict): copies the values out of the dict. Slower than the forms above.
KeyedList(keys, default=0): every key gets the same default value.
KeyedList(other_klist): shares the values list and copies the index.

Reading and writing:

Read with row['a'], row.get('a', default), 'a' in row or row.items(). A list of keys, as in row[['a', 'b']], returns a list of values and skips keys that are missing.
Assigning to an existing key replaces the value in the list.
Assigning to a new key adds the key, and adds the value to the end of the list. If the list is a row of a Daf, that row is now longer than the others.
Deleting a key removes its value from the list.
A KeyedList that shares an index copies it before it adds a key. So adding a key to one row does not give that key to the other rows.

Speed:

Looping over a table as KeyedLists can be faster than looping over dicts, even if you never assign to a row. All the rows share one index, so no row builds its own. The gain grows with the number of columns. In a test with 50,000 rows of 400 columns, reading one field in a loop took 0.047 s with KeyedLists and 0.583 s with dicts. With 5 columns the two were about even.

Because a KeyedList points into the table, assigning to it changes the table. If you need a copy, use to_dict(), or loop with iter_dict() on the Daf.

Examples:

>>> values = [1, 2, 3]
>>> klist = KeyedList(['a', 'b', 'c'], values)
>>> klist['b']
2
>>> klist['b'] = 20
>>> values
[1, 20, 3]
>>> klist.to_dict()
{'a': 1, 'b': 20, 'c': 3}
>>> klist.get('z', 0)
0

A row of a Daf is a KeyedList that points at the row in the table:

>>> from daffodil.daf import Daf
>>> daf = Daf(cols=['x', 'y'], lol=[[1, 2], [3, 4]])
>>> for row in daf.iter_klist():
...     row['y'] = 0
>>> daf

| x  | y  |
| -: | -: |
|  1 |  0 |
|  3 |  0 |

%% daf rows=2; cols=2; keyfield=''; name=''

getitem ¶

__getitem__(key)

Get the value for a key, or a list of values for a list of keys.

Parameters:

Name	Type	Description	Default
`key`	`TKey \| List[TKey]`	A key, or a list of keys.	required

Returns:

Type	Description
`Any`	The value. For a list of keys, a list of values in that order. A key of the list that is
`Any`	missing is skipped, so the list can be shorter.

Raises:

Type	Description
`KeyError`	A single key is not found.
`ValueError`	The key is not a list and cannot be hashed.

Examples:

>>> klist = KeyedList(['a', 'b', 'c'], [1, 2, 3])
>>> klist['b']
2
>>> klist[['c', 'a']]
[3, 1]
>>> klist[['a', 'zz']]
[1]

setitem ¶

__setitem__(key, value)

Set the value for a key, as row[key] = value.

A key that exists gets the new value in the list that this KeyedList points at. If that list is a row of a Daf, the Daf changes. A key that is new is added at the end of the keys, and its value at the end of the list, so a row of a Daf is then longer than the others. The index is copied before it gets a new key, so other KeyedLists that shared it do not get the key.

Parameters:

Name	Type	Description	Default
`key`	`TKey`	The key.	required
`value`	`Any`	The value.	required

Examples:

>>> row = [1, 2]
>>> klist = KeyedList(['a', 'b'], row)
>>> klist['b'] = 20
>>> klist['c'] = 30
>>> row
[1, 20, 30]
>>> list(klist)
['a', 'b', 'c']

delitem ¶

__delitem__(key)

Delete a key and its value, as del row[key].

The value is removed from the list that this KeyedList points at, and the later values move down. If that list is a row of a Daf, the row is then shorter than the others. The index is rebuilt, and this KeyedList no longer shares it.

Parameters:

Name	Type	Description	Default
`key`	`TKey`	The key to delete.	required

Raises:

Type	Description
`KeyError`	The key is not found.

Examples:

>>> row = [1, 2, 3]
>>> klist = KeyedList(['a', 'b', 'c'], row)
>>> del klist['b']
>>> row, list(klist)
([1, 3], ['a', 'c'])

len ¶

__len__()

Get the number of values.

Returns:

Type	Description
`int`	The number of values in the list that this KeyedList points at.

Examples:

>>> len(KeyedList(['a', 'b'], [1, 2]))
2

iter ¶

__iter__()

Loop over the keys, as a dict does.

Returns:

Type	Description
`Iterator[TKey]`	An iterator of the keys, in the order of their values.

Examples:

>>> list(KeyedList(['a', 'b'], [1, 2]))
['a', 'b']

contains ¶

__contains__(key)

Test whether a key is one of the keys, as key in row.

Parameters:

Name	Type	Description	Default
`key`	`object`	The key to look for.	required

Returns:

Type	Description
`bool`	True if the key is found.

Examples:

>>> 'a' in KeyedList(['a', 'b'], [1, 2])
True
>>> 'z' in KeyedList(['a', 'b'], [1, 2])
False

keys ¶

keys()

Get the keys, in the order of their values.

This is a keys view of the index itself. When the index is shared with other KeyedLists, as it is for the rows of a Daf, it is not a copy. For a list of the keys, use list(row.keys()).

Returns:

Type	Description
`KeysView[TKey]`	The keys.

Examples:

>>> KeyedList(['a', 'b'], [1, 2]).keys()
dict_keys(['a', 'b'])

set_values ¶

set_values(new_values)

Point this KeyedList at a different list of values.

This replaces the list that the KeyedList adopted. It does not write into the old list. If the old list was a row of a Daf, the row is not changed, and this KeyedList stops being a view of it. The new list is adopted, not copied. To change the values in the old list, assign to the keys.

Parameters:

Name	Type	Description	Default
`new_values`	`List[Any]`	The new list of values, one for each key.	required

Raises:

Type	Description
`TypeError`	`new_values` is not a list. A tuple is not accepted.
`ValueError`	The list does not have one value for each key.

Examples:

>>> row = [1, 2]
>>> klist = KeyedList(['a', 'b'], row)
>>> klist.set_values([10, 20])
>>> klist.values(), row
([10, 20], [1, 2])
>>> klist.set_values([1])
Traceback (most recent call last):
    ...
ValueError: values length must match keys

values ¶

values(astype=None)

Get the values as a list.

With no astype, or with list, this is the list that the KeyedList adopted, not a copy. If it is a row of a Daf, changing it changes the Daf. With any other astype it is a new list, and the original is not changed. An empty cell, which is NULL in daffodil, is not converted. It stays as the empty string.

Parameters:

Name	Type	Description	Default
`astype`	`Callable \| str \| type \| None`	How to convert each value. A type such as `int`, a function, or one of the names `'int'`, `'str'`, `'float'` and `'bool'`. None does not convert.	`None`

Returns:

Type	Description
`List[Any]`	The values.

Raises:

Type	Description
`ValueError`	`astype` is a name that is not supported, or a value cannot be converted by `int` or `float`. Another converter raises its own error.

Examples:

>>> klist = KeyedList(['a', 'b'], ['1', '2'])
>>> klist.values()
['1', '2']
>>> klist.values(int)
[1, 2]
>>> klist.values('float')
[1.0, 2.0]
>>> KeyedList(['a', 'b'], ['1', '']).values(int)
[1, '']
>>> klist.values() is klist.values()
True
>>> klist.values(int) is klist.values(int)
False
>>> klist.values('date')
Traceback (most recent call last):
    ...
ValueError: astype not supported: date

items ¶

items()

Loop over the (key, value) pairs, as dict.items() does.

This is an iterator, not a view. It can be used once. To keep the pairs, use list(row.items()).

Returns:

Type	Description
`Iterator[Tuple[TKey, Any]]`	An iterator of (key, value) tuples.

Examples:

>>> list(KeyedList(['a', 'b'], [1, 2]).items())
[('a', 1), ('b', 2)]
>>> pairs = KeyedList(['a'], [1]).items()
>>> list(pairs), list(pairs)
([('a', 1)], [])

get ¶

get(key, default=None)

Get the value for a key, or a default if the key is not found.

Parameters:

Name	Type	Description	Default
`key`	`TKey`	The key.	required
`default`	`Any`	What to return if the key is not found.	`None`

Returns:

Type	Description
`Any`	The value, or `default`.

Raises:

Type	Description
`TypeError`	The key cannot be hashed, such as a list. Unlike `row[...]`, a list of keys is not accepted here.

Examples:

>>> klist = KeyedList(['a'], [1])
>>> klist.get('a'), klist.get('z'), klist.get('z', 0)
(1, None, 0)

update ¶

update(other)

Set many keys from a dict or from another KeyedList.

Each key of other is assigned as in row[key] = value. A key that exists gets the new value in the list. A key that is new is added at the end of the keys and of the list. If the list is a row of a Daf, that row is then longer than the others, so use update() only with keys that exist. other is not changed.

Parameters:

Name	Type	Description	Default
`other`	`KeyedList \| Dict[Any, Any]`	A dict or a KeyedList.	required

Examples:

>>> klist = KeyedList(['a', 'b'], [1, 2])
>>> klist.update({'b': 20, 'c': 30})
>>> klist.to_dict()
{'a': 1, 'b': 20, 'c': 30}

to_dict ¶

to_dict()

Make a dict of the keys and values.

The dict is independent of this KeyedList. Later writes to the row do not change it, and changing it does not change the row. The values are not copied deeply, so a list or dict held in a cell is the same object in both.

Returns:

Type	Description
`Dict[TKey, Any]`	A new dict.

Examples:

>>> klist = KeyedList(['a', 'b'], [1, 2])
>>> as_dict = klist.to_dict()
>>> klist['a'] = 99
>>> as_dict
{'a': 1, 'b': 2}

repr ¶

__repr__()

Show the keys and values as a dict.

Returns:

Type	Description
`str`	The text of the dict.

Examples:

>>> KeyedList(['a', 'b'], [1, 2])
{'a': 1, 'b': 2}

bool ¶

__bool__()

return true if there is something in the values list.

to_json ¶

to_json()

Make a JSON string that holds the keys and the values.

The text has the marker __KeyedList__, the keys with their positions under hd, and the values. Every value must be one that json can write. JSON keys are text, so a key that is not a string, such as an int, comes back from from_json() as a string.

Returns:

Type	Description
`str`	The JSON text.

Raises:

Type	Description
`TypeError`	A value cannot be written as JSON.

Examples:

>>> KeyedList(['a', 'b'], [1, 'x']).to_json()
'{"__KeyedList__": true, "hd": {"a": 0, "b": 1}, "values": [1, "x"]}'

from_json `classmethod` ¶

from_json(json_str)

Make a KeyedList from the text that to_json() writes.

Parameters:

Name	Type	Description	Default
`json_str`	`str`	The JSON text.	required

Returns:

Type	Description
`KeyedList`	The new KeyedList. It has its own list of values.

Raises:

Type	Description
`ValueError`	The text is not JSON, or the JSON does not have the `__KeyedList__` marker.

Examples:

>>> klist = KeyedList(['a', 'b'], [1, 'x'])
>>> KeyedList.from_json(klist.to_json()).to_dict()
{'a': 1, 'b': 'x'}
>>> KeyedList.from_json('{"a": 1}')
Traceback (most recent call last):
    ...
ValueError: Invalid JSON string for KeyedList

KeyedList¶

KeyedList ¶

__getitem__ ¶

__setitem__ ¶

__delitem__ ¶

__len__ ¶

__iter__ ¶

__contains__ ¶

keys ¶

set_values ¶

values ¶

items ¶

get ¶

update ¶

to_dict ¶

__repr__ ¶

__bool__ ¶

to_json ¶

from_json classmethod ¶

getitem ¶

setitem ¶

delitem ¶

len ¶

iter ¶

contains ¶

repr ¶

bool ¶

from_json `classmethod` ¶