Data-Transport

A powerful abstraction layer for seamless data communication across diverse systems. data-transport allows you to interact with NoSQL, SQL, and Cloud storage using a consistent interface powered by Pandas and SQLAlchemy.

Why Choose data-transport?

• Unified Interface: Connect to PostgreSQL, MySQL, MongoDB, S3, etc., using the same consistent code.
• Security First: Prevents the dissipation of database connectivity information to protect against security breaches.
• Simplicity & Power: Leverages Pandas DataFrames and SQLAlchemy for intuitive data manipulation.
• Robust Pipelines: Easily integrate pre-processing and post-processing as unified pipelines.
• CLI Integration: Includes a dedicated CLI for registry management and ETL task execution.

Supported Features

Component	Technologies Covered
SQL	PostgreSQL, MySQL, SQL Server, SQLite3+, DuckDB
NoSQL	MongoDB, CouchDB
Warehouse	Apache Iceberg, Apache Drill
Cloud	Nextcloud, S3
Other	Files, RabbitMQ, HTTP

Installation

Install the core package and your desired components:

# Basic installation with default SQL support
pip install data-transport@git+https://github.com/lnyemba/data-transport

# Full suite (SQL, NoSQL, Cloud, Warehouse)
pip install "data-transport[nosql,cloud,warehouse,all]"@git+https://github.com/lnyemba/data-transport.git

Advanced Capabilities

• Automated Pipelines: Seamlessly aggregate functions for data cleaning and transformation.
• Portability: Share notebooks and scripts without exposing raw credentials.
• Scalable Integration: Compatible with environments like Apache Zeppelin, JupyterHub, and SageMaker.

---

Learn More at the Project Website License: MIT