OCR-to-RAG pipeline for life science screenshots

PaddleOCR with preprocessing, scispaCy NER, figure/table detection,
citation extraction, and chunked Markdown output with frontmatter.
Includes watch mode and notebook reprocessing.
This commit is contained in:
2026-07-18 19:34:22 +00:00
parent 012549b4bc
commit a32b7508c7
68 changed files with 8708 additions and 0 deletions

26
.env.example Normal file
View File

@@ -0,0 +1,26 @@
# OCR Pipeline Configuration
# Copy to .env and modify as needed
# Logging
LOG_LEVEL=INFO
LOG_FILE=./data/logs/ocr_pipeline.log
# Input paths (override config.yaml)
INPUT_PATHS=~/Pictures,/mnt/storage3/aman/screenshots
# Processing
WORKERS=4
OCR_ENGINE=paddleocr
USE_GPU=false
# Output
OUTPUT_DIR=./data/ocr_output
ORGANIZE_BY=date_run
# Watch mode
WATCH_ENABLED=true
WATCH_DB_PATH=./data/processed_files.db
# Optional: GROBID for citation parsing
GROBID_ENABLED=false
GROBID_URL=http://localhost:8070