# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.2.0] - 2026-07-21

### Added
- Knowledge graph: `kg_ocr.graph` builds a NetworkX graph (documents, chunks, entities, citations, entity co-occurrence) from pipeline chunk markdown; `kg_ocr.graph.analyzer` provides summaries, top entities/citations, and anomaly detection (low OCR confidence, empty chunks/documents, entity hubs).
- Graph traversal: `chunks_for_entity`, `chunks_for_citation`, `related_entities`, `expand_context` (one-hop CO_OCCURS expansion for graph-RAG), exposed via `ocr-pipeline kg query`.
- `kg_ocr.embeddings.config_loader.KgConfig` for traversal tunables and Neo4j credentials from env.
- `kg_ocr.export`: node-link JSON (lossless round-trip), GraphML for Gephi/Cytoscape, and batched MERGE export to Neo4j via `Neo4jExporter`.
- CLI: `ocr-pipeline kg build|stats|query|export` subcommands with lazy kg_ocr imports.
- Optional extras: `kg` (networkx + txtai + litellm + python-dotenv), `neo4j` (driver), `migrate` (nbformat).
- `init` command: writes a minimal `config.yaml` with platform-correct defaults and checks for Tesseract.
- `demo` command: runs the pipeline on a bundled sample screenshot, prints the output Markdown, and verifies the install in <30s.
- `setup [basic|full]` command: executes install steps with confirmation (`--yes`/`--dry-run`).
- `doctor` command: health check for Python, Tesseract, PaddleOCR, PyTorch backend, table/figure detectors, scispaCy model, config, database, and output directory.
- Lazy settings: `settings` is a proxy; importing the package no longer reads `config.yaml` from cwd.
- Config search paths: `$OCR_PIPELINE_CONFIG` → `./config.yaml` → per-user config dir.
- Optional dependency extras: `paddle`, `tables`, `figures`, `scientific`, `kg`, `full`.
- Engine-aware preprocessing: PaddleOCR reads the (size-capped) original image; Tesseract reads the binarized derivative. Fixes dark-mode OCR quality (confidence 0.83 → 0.95 on test screenshots).
- `data/test_run_full` sample run on 7 screenshots demonstrating full-stack output.
- Tests for citations (19), doctor (15), and startup (12).

### Changed
- Code defaults are now safe: Tesseract OCR, detectors/entities off. The repo's `config.yaml` keeps the full stack for development.
- Detectors default to `enabled: false`; enable per-detector in your config.
- OCR engine default is now `tesseract` (was `paddleocr`).
- `run` results table now shows Skipped count.
- Run directories are created lazily on first write (no empty `run_NNN`).
- `kg_ocr` is now a functional knowledge-graph package (graph/export need only networkx); txtai/litellm imports are lazy so the package imports in slim environments.
- Ruff settings moved to `[tool.ruff.lint]` (silences per-run deprecation warning).
- Pre-commit: bumped ruff, mypy, pre-commit-hooks; dropped the redundant isort hook.
- mypy: added `ignore_missing_imports` for optional deps + `types-PyYAML` in dev extras; `python_version = "3.12"` to match numpy 2.5 stubs.
- README: minimal-style rewrite with collapsible `<details>` sections.

### Fixed
- ISBN regex no longer false-positives on years, URLs, job IDs, or dilutions. Now requires an explicit `ISBN` prefix and matches strict 13- or 10-digit structures.
- `opencv-python-headless` pinned to `<5.0.0`; the 5.0 wheel did not ship a working `cv2` module on Python 3.13.
- `import ocr_pipeline` no longer depends on the caller's cwd.

### Removed
- Dead `src/ocr_pipeline/watch.py` shim (shadowed by the `watch/` package).
- Stub `kg_ocr/cli/` directory (commands now in the main `ocr-pipeline` CLI).
- Untracked `src/ocr_pipeline.egg-info` from git (still gitignored).
- `notebooks/` (personal scratch artifacts; gitignored).
- `nbformat` from core dependencies (moved to `migrate` extra).
- Unused core dependencies: `tqdm`, `scikit-image`, `sqlite-utils`, `xxhash`, `python-slugify`, `python-magic`, `legacy-cgi`.
- Core dependency on `spacy` (moved to `scientific` extra).
- Core dependency on `paddleocr`/`paddlepaddle` (moved to `paddle` extra).
- Core dependency on `torch`/`torchvision`/`transformers`/`timm` (moved to `tables` extra).
- Core dependency on `layoutparser` (moved to `figures` extra).

## [0.1.0] - Initial

Initial release with PaddleOCR + Tesseract fallback, scispaCy NER, layout-aware detectors, citation matching, and Markdown output with YAML frontmatter.
