You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
29 lines
1.7 KiB
Markdown
29 lines
1.7 KiB
Markdown
# QCMS (Quality Control Management System)
|
|
|
|
QCMS is a robust system built to streamline and manage quality control processes.
|
|
|
|
## Powered by data-transport
|
|
|
|
QCMS leverages **data-transport**, a powerful abstraction layer for seamless data communication across diverse systems.
|
|
|
|
### What is data-transport?
|
|
|
|
**data-transport** provides a consistent interface to interact with a wide range of data sources, including:
|
|
|
|
- **SQL Databases:** PostgreSQL, MySQL, SQL Server, SQLite3+, DuckDB
|
|
- **NoSQL Databases:** MongoDB, CouchDB
|
|
- **Cloud Storage & Warehouse:** S3, Nextcloud, Apache Iceberg, Apache Drill
|
|
- **Other Integrations:** Files, RabbitMQ, HTTP
|
|
|
|
### Why use data-transport in QCMS?
|
|
|
|
Integrating data-transport into the QCMS architecture provides several key advantages:
|
|
|
|
- **Unified Data Access:** QCMS can interact with multiple database types (SQL, NoSQL, Cloud) using a consistent code base, simplifying development and maintenance.
|
|
- **Enhanced Security:** It masks raw database connection details, preventing the leak of sensitive credentials within the application.
|
|
- **Seamless Data Manipulation:** By leveraging **Pandas** and **SQLAlchemy**, data-transport allows QCMS to perform complex data transformations and analysis efficiently.
|
|
- **Scalable ETL Pipelines:** It facilitates the creation of robust, reproducible data processing pipelines by treating different storage backends as a unified "transport" layer.
|
|
- **Portability:** It allows for easier sharing of scripts and notebooks across different environments without exposing specific connection strings.
|
|
|
|
For more information on the underlying technology, visit the [data-transport repository](https://github.com/lnyemba/data-transport).
|