improve the OCR pipeline processing and outputs formatting
This commit is contained in:
@@ -15,19 +15,24 @@ Requires-Dist: opencv-python-headless>=4.9.0
|
||||
Requires-Dist: pillow>=10.2.0
|
||||
Requires-Dist: numpy>=1.26.0
|
||||
Requires-Dist: scikit-image>=0.22.0
|
||||
Requires-Dist: paddleocr>=2.7.0
|
||||
Requires-Dist: paddleocr<3.0.0,>=2.7.0
|
||||
Requires-Dist: paddlepaddle>=2.6.0
|
||||
Requires-Dist: pytesseract>=0.3.10
|
||||
Requires-Dist: torch>=2.2.0
|
||||
Requires-Dist: torchvision>=0.17.0
|
||||
Requires-Dist: transformers>=4.38.0
|
||||
Requires-Dist: timm>=1.0.0
|
||||
Requires-Dist: layoutparser>=0.3.0
|
||||
Requires-Dist: langchain-text-splitters>=0.0.2
|
||||
Requires-Dist: spacy>=3.7.0
|
||||
Requires-Dist: nbformat>=5.9.0
|
||||
Requires-Dist: legacy-cgi>=2.6.2
|
||||
Requires-Dist: sqlite-utils>=3.37.0
|
||||
Requires-Dist: watchdog>=3.0.0
|
||||
Requires-Dist: python-slugify>=8.0.0
|
||||
Requires-Dist: xxhash>=3.4.0
|
||||
Requires-Dist: python-magic>=0.4.27
|
||||
Requires-Dist: platformdirs>=4.2.0
|
||||
Provides-Extra: dev
|
||||
Requires-Dist: pytest>=8.0.0; extra == "dev"
|
||||
Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
|
||||
@@ -36,57 +41,45 @@ Requires-Dist: ruff>=0.2.0; extra == "dev"
|
||||
Requires-Dist: mypy>=1.8.0; extra == "dev"
|
||||
Requires-Dist: pre-commit>=3.6.0; extra == "dev"
|
||||
Provides-Extra: grobid
|
||||
Requires-Dist: grobid-client>=0.8.0; extra == "grobid"
|
||||
Provides-Extra: full
|
||||
Requires-Dist: ocr-pipeline[dev]; extra == "full"
|
||||
Requires-Dist: ocr-pipeline[grobid]; extra == "full"
|
||||
|
||||
# OCR Pipeline for Life Science Screenshots
|
||||
|
||||
A production-ready OCR pipeline that extracts text from scientific screenshots and converts them into RAG-ready Markdown files with rich metadata.
|
||||
Turns scientific screenshots into RAG-ready Markdown, with metadata attached.
|
||||
|
||||
## Features
|
||||
- OCR via PaddleOCR, falls back to Tesseract
|
||||
- Preprocessing: deskew, denoise, CLAHE, line removal
|
||||
- Figure/table detection
|
||||
- Entity extraction with scispaCy
|
||||
- Citation matching
|
||||
- Chunking for embedding
|
||||
|
||||
- **Multi-engine OCR**: PaddleOCR (primary) with Tesseract fallback
|
||||
- **Image preprocessing**: Deskewing, denoising, CLAHE contrast enhancement, line removal
|
||||
- **Parallel processing**: Multi-process worker pool for throughput
|
||||
- **Scientific entity recognition**: Genes, proteins, chemicals, species, diseases, cell lines via scispaCy
|
||||
- **Figure & table detection**: LayoutParser + Table Transformer
|
||||
- **Citation extraction**: DOI, PMID, arXiv, PMC, ISBN via regex + optional GROBID
|
||||
- **Semantic chunking**: LangChain recursive splitter optimized for scientific text
|
||||
- **RAG-ready output**: Markdown with YAML frontmatter (source, hash, timestamp, entities, confidence)
|
||||
- **Watch mode**: File system monitoring with SQLite persistence for incremental processing
|
||||
- **Notebook migration**: Reprocess screenshots from existing Jupyter notebooks
|
||||
- **Life-science focused**: Handles scientific notation, units, gene symbols, chemical formulas
|
||||
|
||||
## Quick Start
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
# Install with uv (recommended)
|
||||
uv sync
|
||||
|
||||
# Or with pip
|
||||
pip install -e .
|
||||
|
||||
# Install pre-commit hooks
|
||||
uv sync # or: pip install -e .
|
||||
pre-commit install
|
||||
|
||||
# Run on your screenshots
|
||||
ocr-pipeline run
|
||||
|
||||
# Watch for new screenshots
|
||||
ocr-pipeline watch
|
||||
|
||||
# Reprocess from notebook
|
||||
ocr-pipeline reprocess ocr_sc.ipynb
|
||||
|
||||
# Check status
|
||||
ocr-pipeline run # process screenshots
|
||||
ocr-pipeline watch # watch a folder
|
||||
ocr-pipeline reprocess ocr_sc.ipynb # pull screenshots out of a notebook
|
||||
ocr-pipeline status
|
||||
```
|
||||
|
||||
## Configuration
|
||||
## Docker
|
||||
|
||||
Edit `config.yaml`:
|
||||
```bash
|
||||
docker build -t ocr-pipeline .
|
||||
|
||||
docker run -v ~/Pictures:/data/screenshots -v ./data:/app/data ocr-pipeline run
|
||||
docker run -v ~/Pictures:/data/screenshots -v ./data:/app/data ocr-pipeline watch
|
||||
```
|
||||
|
||||
## Config
|
||||
|
||||
<details>
|
||||
<summary><code>config.yaml</code></summary>
|
||||
|
||||
```yaml
|
||||
input:
|
||||
@@ -120,11 +113,11 @@ detectors:
|
||||
|
||||
entities:
|
||||
enabled: true
|
||||
model: "en_core_sci_lg" # scispaCy large model
|
||||
model: "en_core_sci_lg"
|
||||
|
||||
citations:
|
||||
regex_enabled: true
|
||||
grobid_enabled: false # Set true if running GROBID server
|
||||
grobid_enabled: false # set true if you're running a GROBID server
|
||||
|
||||
chunking:
|
||||
chunk_size: 1000
|
||||
@@ -153,9 +146,11 @@ watch:
|
||||
db_path: "./data/processed_files.db"
|
||||
```
|
||||
|
||||
## Output Format
|
||||
</details>
|
||||
|
||||
Each chunk produces a Markdown file with YAML frontmatter:
|
||||
## Output
|
||||
|
||||
Each chunk is a Markdown file with YAML frontmatter:
|
||||
|
||||
```markdown
|
||||
---
|
||||
@@ -178,9 +173,9 @@ total_chunks: 2
|
||||
## Figures
|
||||
|
||||
### Figure 1
|
||||
- **BBox**: [100, 200, 800, 600]
|
||||
- **Confidence**: 0.92
|
||||
- **Caption**: "Western blot showing BRCA1 expression..."
|
||||
- BBox: [100, 200, 800, 600]
|
||||
- Confidence: 0.92
|
||||
- Caption: "Western blot showing BRCA1 expression..."
|
||||
|
||||
## Detected Entities
|
||||
|
||||
@@ -205,72 +200,33 @@ Lane 4: BRCA1 KO + pBRCA1-C61G mutant
|
||||
Anti-BRCA1 (1:1000), Anti-β-actin (1:5000)
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
Files land in `data/ocr_output/<date>/run_NNN/`.
|
||||
|
||||
```
|
||||
data/ocr_output/
|
||||
├── 2025-01-15/
|
||||
│ ├── run_001/
|
||||
│ │ ├── SCR-20250115-gel_chunk_000.md
|
||||
│ │ └── SCR-20250115-gel_chunk_001.md
|
||||
│ └── run_002/
|
||||
│ └── ...
|
||||
└── 2025-01-16/
|
||||
└── run_001/
|
||||
└── ...
|
||||
```
|
||||
## Life science specifics
|
||||
|
||||
## Docker
|
||||
Gene/protein names go through scispaCy's `en_core_sci_lg`. Chemical formulas and units (µM, ng/mL, kb/Mb/Gb, °C, ×g) get normalized, scientific notation gets cleaned up (`1.5×10⁻³` → `1.5×10^-3`), and gel/blot figures get their captions pulled out separately. Citations are matched by regex for DOI, PMID, arXiv, PMC, and ISBN.
|
||||
|
||||
```bash
|
||||
# Build
|
||||
docker build -t ocr-pipeline .
|
||||
## Models
|
||||
|
||||
# Run once
|
||||
docker run -v ~/Pictures:/data/screenshots -v ./data:/app/data ocr-pipeline run
|
||||
First run downloads what it needs, cached in `~/.cache/ocr_pipeline/`:
|
||||
|
||||
# Watch mode
|
||||
docker run -v ~/Pictures:/data/screenshots -v ./data:/app/data ocr-pipeline watch
|
||||
```
|
||||
|
||||
## Life Science Optimizations
|
||||
|
||||
| Feature | Implementation |
|
||||
|---------|----------------|
|
||||
| Gene/Protein names | scispaCy `en_core_sci_lg` NER |
|
||||
| Chemical formulas | Regex + unit normalization (µM, ng/mL, kb, etc.) |
|
||||
| Scientific notation | `1.5×10⁻³` → `1.5×10^-3` |
|
||||
| Gel/blot lanes | Figure detection + caption extraction |
|
||||
| Citations | DOI, PMID, arXiv, PMC, ISBN patterns |
|
||||
| Units | µM, ng/mL, kb/Mb/Gb, °C, ×g, etc. |
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
# Install dev dependencies
|
||||
uv sync --dev
|
||||
|
||||
# Run tests
|
||||
pytest
|
||||
|
||||
# Lint
|
||||
ruff check .
|
||||
ruff format .
|
||||
|
||||
# Type check
|
||||
mypy src/
|
||||
```
|
||||
|
||||
## Model Downloads
|
||||
|
||||
First run downloads models automatically:
|
||||
- PaddleOCR detection/recognition models (~200MB)
|
||||
- PaddleOCR models (~200MB)
|
||||
- scispaCy `en_core_sci_lg` (~800MB)
|
||||
- Table Transformer (~500MB)
|
||||
- LayoutParser PubLayNet (~300MB)
|
||||
|
||||
Cache location: `~/.cache/ocr_pipeline/`
|
||||
## Runtime requirements and health
|
||||
|
||||
## License
|
||||
The core pipeline can run with Tesseract alone. PaddleOCR, scientific NER, figure detection, and table detection are optional capabilities with heavyweight, platform-specific dependencies. The pipeline records the OCR engine actually used and the entity-extraction backend in generated frontmatter; inspect them after each run rather than assuming configured models loaded.
|
||||
|
||||
- **PaddleOCR:** the project pins the legacy 2.x API used by the pipeline. Install the locked environment with `uv sync`; a startup fallback to Tesseract is logged when Paddle cannot initialize.
|
||||
- **Scientific NER:** install a compatible `scispacy` distribution and the separately distributed `en_core_sci_lg` model before enabling production scientific NER. Without it, the pipeline uses its conservative regex fallback and marks the backend accordingly.
|
||||
- **Figures:** `layoutparser`'s `Detectron2LayoutModel` requires a Detectron2 build matching your Torch/Python platform. It is intentionally not forced as a universal dependency because no single wheel supports every platform.
|
||||
- **Tables:** the Table Transformer model is downloaded by Transformers on first use; ensure the selected model's optional dependencies (including `timm`, when required by that model revision) are installed in the runtime image.
|
||||
|
||||
Use `ocr-pipeline config` to verify effective settings. A missing optional model is logged and produces empty results for that detector instead of failing a complete batch.
|
||||
|
||||
## TODO
|
||||
|
||||
- [ ] Build a knowledge graph from extracted entities/citations
|
||||
|
||||
MIT
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
README.md
|
||||
pyproject.toml
|
||||
setup.py
|
||||
src/ocr_pipeline/__init__.py
|
||||
src/ocr_pipeline/cli.py
|
||||
src/ocr_pipeline/config.py
|
||||
@@ -20,6 +21,7 @@ src/ocr_pipeline/detectors/tables.py
|
||||
src/ocr_pipeline/ocr/__init__.py
|
||||
src/ocr_pipeline/ocr/engine.py
|
||||
src/ocr_pipeline/ocr/parallel.py
|
||||
src/ocr_pipeline/ocr/preprocess.py
|
||||
src/ocr_pipeline/output/__init__.py
|
||||
src/ocr_pipeline/output/markdown.py
|
||||
src/ocr_pipeline/postprocess/__init__.py
|
||||
@@ -32,4 +34,8 @@ src/ocr_pipeline/utils/logging.py
|
||||
src/ocr_pipeline/utils/migrate.py
|
||||
src/ocr_pipeline/watch/__init__.py
|
||||
src/ocr_pipeline/watch/watcher.py
|
||||
tests/test_graph.py
|
||||
tests/test_indexer.py
|
||||
tests/test_ocr.py
|
||||
tests/test_output.py
|
||||
tests/test_postprocess.py
|
||||
@@ -9,19 +9,24 @@ opencv-python-headless>=4.9.0
|
||||
pillow>=10.2.0
|
||||
numpy>=1.26.0
|
||||
scikit-image>=0.22.0
|
||||
paddleocr>=2.7.0
|
||||
paddleocr<3.0.0,>=2.7.0
|
||||
paddlepaddle>=2.6.0
|
||||
pytesseract>=0.3.10
|
||||
torch>=2.2.0
|
||||
torchvision>=0.17.0
|
||||
transformers>=4.38.0
|
||||
timm>=1.0.0
|
||||
layoutparser>=0.3.0
|
||||
langchain-text-splitters>=0.0.2
|
||||
spacy>=3.7.0
|
||||
nbformat>=5.9.0
|
||||
legacy-cgi>=2.6.2
|
||||
sqlite-utils>=3.37.0
|
||||
watchdog>=3.0.0
|
||||
python-slugify>=8.0.0
|
||||
xxhash>=3.4.0
|
||||
python-magic>=0.4.27
|
||||
platformdirs>=4.2.0
|
||||
|
||||
[dev]
|
||||
pytest>=8.0.0
|
||||
@@ -33,7 +38,5 @@ pre-commit>=3.6.0
|
||||
|
||||
[full]
|
||||
ocr-pipeline[dev]
|
||||
ocr-pipeline[grobid]
|
||||
|
||||
[grobid]
|
||||
grobid-client>=0.8.0
|
||||
|
||||
Reference in New Issue
Block a user