uv sync
pre-commit install

# Smoke test a small directory with the lightweight OCR path
uv run ocr-pipeline run \
  --input-dir ~/Pictures/test_screenshots \
  --output-dir /tmp/ocr-output \
  --engine tesseract \
  --workers 1 \
  --exclude "*.photoslibrary/*"

uv run ocr-pipeline watch
uv run ocr-pipeline reprocess ocr_sc.ipynb
uv run ocr-pipeline status

Use --force to reprocess files already recorded in the processing database. The --exclude option is repeatable and prevents recursive scans from entering bundles such as macOS .photoslibrary directories.

Scientific NER environment

en_core_sci_lg is distributed separately from scispaCy and is not installed by uv sync. The compatible scispaCy 0.5.4 release requires spaCy 3.7.x, so install it in a dedicated Python 3.11 environment rather than mixing it with the main Python 3.13 environment:

uv venv --python 3.11 .venv-scispacy
source .venv-scispacy/bin/activate
uv pip install "spacy>=3.7,<3.8" "scispacy==0.5.4" click
uv pip install https://s3-us-west-2.amazonaws.com/ai2-s2-scispacy/releases/v0.5.4/en_core_sci_lg-0.5.4.tar.gz
uv pip install -e .
uv pip install timm

# Confirm the model and command use this same environment
.venv-scispacy/bin/python -c "import spacy; spacy.load('en_core_sci_lg'); print('scispaCy OK')"
.venv-scispacy/bin/ocr-pipeline run --help

If the two environments are kept separate, invoke the pipeline with the explicit .venv-scispacy/bin/ocr-pipeline path; uv run selects the project's default .venv.

Docker

docker build -t ocr-pipeline .

docker run -v ~/Pictures:/data/screenshots -v ./data:/app/data ocr-pipeline run
docker run -v ~/Pictures:/data/screenshots -v ./data:/app/data ocr-pipeline watch

Config

config.yaml

input:
  paths: ["~/Pictures", "/mnt/storage3/aman/screenshots"]
  patterns: ["SCR-*.png", "*.jpg", "*.jpeg", "*.tiff"]
  recursive: true
  exclude_patterns: ["*.photoslibrary/*"]

ocr:
  engine: "paddleocr"  # paddleocr | tesseract | auto
  languages: ["en", "latin"]
  use_gpu: false
  preprocess:
    deskew: true
    denoise: true
    clahe: true
    adaptive_threshold: true
    remove_lines: true

processing:
  workers: 4
  batch_size: 10
  retry_attempts: 2

detectors:
  figures:
    enabled: true
    confidence_threshold: 0.7
  tables:
    enabled: true
    confidence_threshold: 0.7

entities:
  enabled: true
  model: "en_core_sci_lg"

citations:
  regex_enabled: true
  grobid_enabled: false  # set true if you're running a GROBID server

chunking:
  chunk_size: 1000
  chunk_overlap: 200

output:
  base_directory: "./data/ocr_output"
  organize_by: "date_run"  # date_run | source_dir | flat
  write_consolidated: true
  consolidated_filename: "all_ocr.md"
  frontmatter:
    - source_path
    - source_hash
    - timestamp
    - ocr_engine
    - ocr_confidence_mean
    - language
    - detected_entities
    - entity_extraction_backend
    - has_figures
    - has_tables
    - citations_found
    - chunk_index
    - total_chunks

watch:
  enabled: true
  debounce_seconds: 5
  db_path: "./data/processed_files.db"

Output

Each chunk is a Markdown file with YAML frontmatter:

---
source_path: "/Users/Aman/Pictures/SCR-20250115-gel.png"
source_hash: "a1b2c3d4e5f6..."
timestamp: "2025-01-15T10:30:00Z"
ocr_engine: "paddleocr"
ocr_confidence_mean: 0.91
language: "en"
detected_entities: ["GENE", "PROTEIN", "CHEMICAL"]
has_figures: true
has_tables: false
citations_found: ["DOI:10.1038/nature12345", "PMID:12345678"]
chunk_index: 0
total_chunks: 2
---

# Screenshot: SCR-20250115-gel.png

## Figures

### Figure 1
- BBox: [100, 200, 800, 600]
- Confidence: 0.92
- Caption: "Western blot showing BRCA1 expression..."

## Detected Entities

- BRCA1
- CRISPR
- β-actin

## Citations

- DOI: 10.1038/nature12345
- PMID: 12345678

## Extracted Text

**Western Blot Analysis of BRCA1 Expression**

Lane 1: WT control
Lane 2: BRCA1 KO (CRISPR)
Lane 3: BRCA1 KO + pBRCA1-WT rescue
Lane 4: BRCA1 KO + pBRCA1-C61G mutant

Anti-BRCA1 (1:1000), Anti-β-actin (1:5000)

Files land in data/ocr_output/<date>/run_NNN/. Individual chunk files are retained for RAG indexing. Set output.write_consolidated: true to also write all_ocr.md containing one frontmatter block and all source text grouped by image.

Life science specifics

Gene/protein names go through scispaCy's en_core_sci_lg. Chemical formulas and units (µM, ng/mL, kb/Mb/Gb, °C, ×g) get normalized, scientific notation gets cleaned up (1.5×10⁻³ → 1.5×10^-3), and gel/blot figures get their captions pulled out separately. Citations are matched by regex for DOI, PMID, arXiv, PMC, and ISBN.

Models

First run downloads what it needs, cached in ~/.cache/ocr_pipeline/:

PaddleOCR models (~200MB)
scispaCy en_core_sci_lg (~800MB)
Table Transformer (~500MB)
LayoutParser PubLayNet (~300MB)

Runtime requirements and health

The core pipeline can run with Tesseract alone. PaddleOCR, scientific NER, figure detection, and table detection are optional capabilities with heavyweight, platform-specific dependencies. The pipeline records the OCR engine actually used and the entity-extraction backend in generated frontmatter; inspect them after each run rather than assuming configured models loaded.

PaddleOCR: the project pins the legacy 2.x API used by the pipeline. Install the locked environment with uv sync; a startup fallback to Tesseract is logged when Paddle cannot initialize.
Scientific NER: install a compatible scispacy distribution and the separately distributed en_core_sci_lg model before enabling production scientific NER. Without it, the pipeline uses its conservative regex fallback and marks the backend accordingly.
Figures: layoutparser's Detectron2LayoutModel requires a Detectron2 build matching your Torch/Python platform. It is intentionally not forced as a universal dependency because no single wheel supports every platform.
Tables: the Table Transformer model is downloaded by Transformers on first use; ensure the selected model's optional dependencies (including timm, when required by that model revision) are installed in the runtime image.

Use ocr-pipeline config to verify effective settings. A missing optional model is logged and produces empty results for that detector instead of failing a complete batch.

TODO

Build a knowledge graph from extracted entities/citations

README.md Unescape Escape

OCR Pipeline for Life Science Screenshots

Quick start