Skip to content

About

Python Filesystem interface for non-filesystem data sources

Topics

Resources

Code of conduct

Stars

6 stars

Watchers

2 watching

Forks

Repository files navigation

fsspec-data

Convert tabular formats and reconcile Arrow schemas over fsspec.

Build Status codecov License PyPI

fsspec-data plans schema mappings and executes them over Arrow record batches. Codecs handle Arrow IPC, Parquet, CSV, and JSONL. A read-only chained filesystem exposes converted files; a separate converter registry handles resources such as Xarray datasets and Zarr stores.

Start here

Convert CSV identifiers to Parquet walks through planning, batch iteration, and encoded output.

How-to guides

For application work, connect a batch producer or consumer, convert a file through an fsspec chain, or resolve schemas from files and registries. Convert Xarray and Zarr resources covers native resource routes and converter registration.

Reference and explanation

The interchange API reference covers codecs, policies, limits, and planning. Schema and converter reference describes resolution and resource routes. Read Interchange design for the boundaries between schemas, data, and consumers.

About

Python Filesystem interface for non-filesystem data sources

Topics

Resources

Code of conduct

Stars

6 stars

Watchers

2 watching

Forks

Releases

Used by

Contributors

Languages