Daf overview¶
Daf ¶
A table of data, stored as a list of rows.
Daf is a small, fast, pure Python table. Use it to read, reshape and write 2-D data without the weight of pandas. It is not meant for heavy numeric work.
The rows are a list of lists named lol. A row holds only values. The column
names are kept once, in a header dict named hd that maps each name to its
position. That makes rows cheap to build, append and copy.
A Daf may have a keyfield. This is a column, or a tuple of columns, whose
values name the rows. The key index is built the first time a key is needed.
Methods that add, remove or reorder rows keep it up to date, or mark it to be rebuilt.
Changing a key value inside a row does not. See __setitem__().
A missing value is NULL, which is the empty string. It prints as nothing.
A cell can hold any Python object, such as a number, text, a list, a dict, a set or
an object of your own. Nothing limits what a cell can hold. The dtypes are not a
schema that checks or forces the cells. They say how to convert columns, as when
apply_dtypes() turns the text of a CSV file into numbers. A Daf built with dtypes
keeps its cells as they are.
Rows are returned as dicts or as KeyedList objects. See retmode and itermode.
Examples:
>>> d = Daf(lol=[[1, 'a'], [2, 'b']], cols=['id', 'v'], keyfield='id')
>>> d.columns()
['id', 'v']
>>> d.select_record(2)
{'id': 2, 'v': 'b'}
>>> d.shape()
(2, 2)
>>> e = Daf(lol=[[[1, 2], {'k': 1}]], cols=['a', 'b'], dtypes={'a': int, 'b': str})
>>> e.iloc(0)
{'a': [1, 2], 'b': {'k': 1}}
Names in the signatures¶
The signatures use short names for types. Each is T_ and then letters for the type, read
from the outside in. They are defined in daffodil.lib.daf_types.
| Letter | Means |
|---|---|
l |
a list |
d |
a dict with str keys |
id |
a dict with int keys |
t |
a tuple |
s, i, f, b |
str, int, float, bool |
a |
any value |
o |
"of", between two parts |
So T_ls is a list of str, T_da is a dict of any values, T_lola is a list of lists of any
values, the rows of a Daf, and T_dols is a dict of lists of str. A few names do not follow
the scheme:
| Name | Means |
|---|---|
T_cs |
column names: a list of str, the keys of a dict, or a dict |
T_ma |
a row as a dict or a KeyedList |
T_klodaf |
a KeyedList of Dafs, one for each group, as groupby() gives |
T_dtype_dict |
a dict of column name to type, as dtypes |
T_lor |
a list of ranges |
A default of _MISSING means that no value was given. It lets None be a real value that you
pass. For example, to_value() with no default raises for an empty selection, and with
default=None it returns None.
Constructor¶
__init__ ¶
__init__(
lol=None,
hd=None,
kd=None,
cols=None,
dtypes=None,
schema=None,
keyfield="",
name="",
use_copy=False,
disp_cols=None,
retmode="obj",
itermode="dict",
attrs=None,
)
Create a Daf from rows, column names and options.
Every argument is optional, so Daf() makes an empty table. The usual call
gives the rows and the column names.
The rows are not copied. The lol list you pass in becomes the data of the
Daf, so changing one changes the other. The same holds for hd, kd and
attrs. Pass use_copy=True to deep copy lol first.
The dtypes can only name columns. If dtypes has a name that is not a column, the Daf
keeps a new dict without it, and your dict is not changed. A dict that is already right is adopted.
If cols is given, it sets the column names. If it is not given, the keys of
dtypes set them. Otherwise hd is used. A name that is empty or repeated
is made unique, so ['a', 'a'] becomes ['a', 'a_1'].
With no names at all, the Daf has no columns and lol is only a list of rows.
Call set_cols() to name them later.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
lol
|
T_lola | None
|
Rows, as a list of lists. Adopted, not copied. |
None
|
hd
|
T_di | None
|
Header dict that maps column name to position. |
None
|
kd
|
Dict[str | int, int] | None
|
Key index to adopt when no keyfield is set. |
None
|
cols
|
T_cs | None
|
Column names. These win over |
None
|
dtypes
|
T_dtype_dict | None
|
Type for each column, used when converting, as from the text of a CSV file. It does not limit what a cell can hold. Entries for names that are not columns are left out. |
None
|
schema
|
type | Daf | None
|
A |
None
|
keyfield
|
str | int | T_ta | T_la
|
Column, or tuple or list of columns, whose values identify rows. Each must be a column. See |
''
|
name
|
str
|
Free text name of this Daf. |
''
|
use_copy
|
bool
|
If True, deep copy |
False
|
disp_cols
|
T_cs | None
|
Column names to show when the Daf is printed. |
None
|
retmode
|
str
|
Whether a one cell result is returned as a Daf or a bare value. |
'obj'
|
itermode
|
str
|
Whether iteration yields dicts or KeyedList objects. |
'dict'
|
attrs
|
T_da | None
|
Free form dict of extra information. Adopted, not copied. |
None
|
A keyfield must be made of columns, or the Daf raises KeyError. A Daf with no column
names yet can still be given one, for the columns that come later, as in
Daf(keyfield='id') followed by append() of dicts.
Raises:
| Type | Description |
|---|---|
TypeError
|
|
KeyError
|
The |
Examples:
>>> d = Daf(lol=[[1, 'a'], [2, 'b']], cols=['id', 'v'], keyfield='id')
>>> d.num_rows()
2
>>> Daf(lol=[[1, 2]], cols=['a', 'a']).columns()
['a', 'a_1']
Class constants¶
TYPE_MAP
class-attribute
instance-attribute
¶
TYPE_MAP = {
"int": int,
"float": float,
"str": str,
"bool": bool,
"list": list,
"dict": dict,
}