Our knowledge system systematically ingests, processes, and indexes open-access life science publications. It supports internal research by providing precise question-answering and efficient retrieval from a continuously updated repository of scientific literature
Export sales data from Google Sheet to a relational DBSM
The main purpose of this repository is to build the pipeline for training of regression models and predict the compressive strength of concrete to reduce the risk and cost involved in discarding the concrete structures when the concrete cube test fails.
ECGO Brands + Flavors Streamlit App Built an interactive Streamlit-based analytics application to transform COCO-format JSON outputs from Roboflow into structured CSV reports and visual insights. Developed a custom Python (Pandas) parsing pipeline to extract data.
Python-based Data Engineering project using Databricks Asset Bundles, PySpark, Unity Catalog, Python Wheel packaging, automated testing, Dev Containers, and GitHub Actions CI/CD for DEV and PROD deployments
Resource for ETL & Data Ingestion program using Apache Airflow
This project implements a real-time data pipeline using Apache Airflow, Kafka, Apache Spark, and Delta Lake. It supports both batch (Coldpath) and real-time (Hotpath) data ingestion, processing, and storage. Airflow is used for orchestrating the data workflows.
An application of my Centipede framework to watch 4Chan for any potentially threatening behavior.
RAG-based chatbot that answers queries from university Common Data Set (CDS) PDFs and working on generating comparison tables for tuition, fees, and housing.
To associate your repository with the dataingestion topic, visit your repo's landing page and select "manage topics."