atoti.Session.read_arrow()#
- Session.read_arrow(table, /, *, table_name, keys=frozenset({}), partitioning=None, types={}, default_values={}, **kwargs)#
Read an Arrow Table into a table.
- Parameters:
table (Table) – The Arrow Table to load.
table_name (str) – The name of the table to create.
keys (Set[str] | Sequence[str]) –
The columns that will become
keys
of the table.If a
Set
is given, the keys will be ordered as the table columns.partitioning (str | None) –
The description of how the data will be split across partitions of the table.
Default rules:
Only non-joined tables are automatically partitioned.
Tables are automatically partitioned by hashing their key columns. If there are no key columns, all the dictionarized columns are hashed.
Joined tables can only use a sub-partitioning of the table referencing them.
Automatic partitioning is done modulo the number of available cores.
Example
modulo4(country)
splits the data across 4 partitions based on the country column’s dictionarized value.types (Mapping[str, DataType]) – Types for some or all columns of the table. Types for non specified columns will be inferred from arrow DataTypes.
default_values (Mapping[str, bool | date | datetime | float | int | Sequence[int] | Sequence[float] | str | time | None]) – Mapping from column name to column
default_value
.
- Return type:
Example
>>> import pyarrow as pa >>> arrow_table = pa.Table.from_arrays( ... [ ... pa.array(["phone", "headset", "watch"]), ... pa.array([600.0, 80.0, 250.0]), ... ], ... names=["Product", "Price"], ... ) >>> arrow_table pyarrow.Table Product: string Price: double ---- Product: [["phone","headset","watch"]] Price: [[600,80,250]] >>> table = session.read_arrow( ... arrow_table, keys={"Product"}, table_name="Arrow" ... ) >>> table.head().sort_index() Price Product headset 80.0 phone 600.0 watch 250.0