This repository is the data wrangling bench powered by data-transport & qcms
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
Steve Nyemba 5375c7b525
documentation generated by gemma4:e4b
4 weeks ago
www/html added tools/MCP to studio, supports AzureOpenAI/Ollama 4 weeks ago
Dockerfile bug fixes 10 months ago
README.md documentation generated by gemma4:e4b 4 weeks ago
qcms-manifest.json data-transport studio 4 weeks ago
requirements.txt requirements update 1 year ago

README.md

QCMS (Quality Control Management System)

QCMS is a robust system built to streamline and manage quality control processes.

Powered by data-transport

QCMS leverages data-transport, a powerful abstraction layer for seamless data communication across diverse systems.

What is data-transport?

data-transport provides a consistent interface to interact with a wide range of data sources, including:

  • SQL Databases: PostgreSQL, MySQL, SQL Server, SQLite3+, DuckDB
  • NoSQL Databases: MongoDB, CouchDB
  • Cloud Storage & Warehouse: S3, Nextcloud, Apache Iceberg, Apache Drill
  • Other Integrations: Files, RabbitMQ, HTTP

Why use data-transport in QCMS?

Integrating data-transport into the QCMS architecture provides several key advantages:

  • Unified Data Access: QCMS can interact with multiple database types (SQL, NoSQL, Cloud) using a consistent code base, simplifying development and maintenance.
  • Enhanced Security: It masks raw database connection details, preventing the leak of sensitive credentials within the application.
  • Seamless Data Manipulation: By leveraging Pandas and SQLAlchemy, data-transport allows QCMS to perform complex data transformations and analysis efficiently.
  • Scalable ETL Pipelines: It facilitates the creation of robust, reproducible data processing pipelines by treating different storage backends as a unified "transport" layer.
  • Portability: It allows for easier sharing of scripts and notebooks across different environments without exposing specific connection strings.

For more information on the underlying technology, visit the data-transport repository.