This framework allows read/write and ETL against many SQL, Cloud, NoSQL databases, and other persistent data stores. Additional features include support for user-defined plugin pre/post processing functions
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
Go to file
Steve L. Nyemba 081ed080d7
bug fixes for version 1.0.8, streamlining interface
5 years ago
transport bug fixes for version 1.0.8, streamlining interface 5 years ago
.gitignore documentation and housekeeping work 5 years ago
README.md documentation and housekeeping work 5 years ago
requirements.txt S3 Requirments file 7 years ago
setup.py bug fixes for version 1.0.8, streamlining interface 5 years ago

README.md

Introduction

This project implements an abstraction of objects that can have access to a variety of data stores, implementing read/write with a simple interface against specific various data-sources. The supported data sources implement functionalities against :

- Rabbitmq-server
- Couchdb-server
- Mongodb-server
- Http Session : {csv,tab,pipe,sql}
- Disk{Reader|Writer} : csv, tab, pipe, sql on disk

Such an interface is used to facilitate data transport in and out of a store for whatever an application may need (log, session management, ...)

Installation

Within the virtual environment perform the following command:

pip install git+https://dev.the-phi.com/git/steve/data-transport.git

Binaries and eggs will be provided later on

Usage

The basic usage revolves around a factory class (to be a singleton)

import transport
from transport import factory
#
# importing a mongo reader
args = {"host":"<host>:<port>","dbname":"<database>","doc":"<doc_id>",["username":"<username>","password":"<password>"]}
mreader = factory.instance(type='mongo.MonoReader',args=args)
#
# reading a document and executing a view
#
document    = mreader.read()
result      = mreader.view(name)
#
# importing a couchdb reader
args = {"url":"<http://host>:<port>","dbname":"<database>","doc":"<doc_id>","username":"<username>","password":"<password>"}
creader     = factory.instance(type='couch.CouchReader',args=args)

#
# Reading a document and executing a view
#
document    = dreader.read()    
result      = couchdb.view(id='<design_doc_id>',view_name=<view_name',<key=value|keys=values>)