kg: graph build, traversal queries, neo4j export - kg_ocr.graph builds a networkx graph (docs, chunks, entities, citations, co-occurrence) from chunk markdown - analyzer for summaries, top entities/citations, anomaly checks - traversal: chunks_for_entity/citation, related_entities, expand_context - export: JSON round-trip, GraphML, batched MERGE into neo4j - new CLI: ocr-pipeline kg build|stats|query|export - lazy kg_ocr imports, networkx/neo4j behind extras - dropped dead watch.py shim, added KgConfig stub - trimmed README, updated TODO
Some checks failed
tests / core (macos-latest, py3.11) (push) Has been cancelled
tests / core (macos-latest, py3.12) (push) Has been cancelled
tests / core (macos-latest, py3.13) (push) Has been cancelled
tests / core (ubuntu-latest, py3.11) (push) Has been cancelled
tests / core (ubuntu-latest, py3.12) (push) Has been cancelled
tests / core (ubuntu-latest, py3.13) (push) Has been cancelled
tests / doctor CLI smoke test (push) Has been cancelled
Some checks failed
tests / core (macos-latest, py3.11) (push) Has been cancelled
tests / core (macos-latest, py3.12) (push) Has been cancelled
tests / core (macos-latest, py3.13) (push) Has been cancelled
tests / core (ubuntu-latest, py3.11) (push) Has been cancelled
tests / core (ubuntu-latest, py3.12) (push) Has been cancelled
tests / core (ubuntu-latest, py3.13) (push) Has been cancelled
tests / doctor CLI smoke test (push) Has been cancelled
This commit is contained in:
@@ -8,6 +8,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- Knowledge graph: `kg_ocr.graph` builds a NetworkX graph (documents, chunks, entities, citations, entity co-occurrence) from pipeline chunk markdown; `kg_ocr.graph.analyzer` provides summaries, top entities/citations, and anomaly detection (low OCR confidence, empty chunks/documents, entity hubs).
|
||||
- Graph traversal: `chunks_for_entity`, `chunks_for_citation`, `related_entities`, `expand_context` (one-hop CO_OCCURS expansion for graph-RAG), exposed via `ocr-pipeline kg query`.
|
||||
- `kg_ocr.embeddings.config_loader.KgConfig` for traversal tunables and Neo4j credentials from env.
|
||||
- `kg_ocr.export`: node-link JSON (lossless round-trip), GraphML for Gephi/Cytoscape, and batched MERGE export to Neo4j via `Neo4jExporter`.
|
||||
- CLI: `ocr-pipeline kg build|stats|query|export` subcommands with lazy kg_ocr imports.
|
||||
- Optional extras: `kg` (networkx + txtai + litellm + python-dotenv), `neo4j` (driver).
|
||||
- `init` command: writes a minimal `config.yaml` with platform-correct defaults and checks for Tesseract.
|
||||
- `demo` command: runs the pipeline on a bundled sample screenshot, prints the output Markdown, and verifies the install in <30s.
|
||||
- `setup [basic|full]` command: executes install steps with confirmation (`--yes`/`--dry-run`).
|
||||
@@ -25,7 +31,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- OCR engine default is now `tesseract` (was `paddleocr`).
|
||||
- `run` results table now shows Skipped count.
|
||||
- Run directories are created lazily on first write (no empty `run_NNN`).
|
||||
- `kg_ocr` package marked experimental; deps under `--extra kg`, helpful ImportError on bare import.
|
||||
- `kg_ocr` is now a functional knowledge-graph package (graph/export need only networkx); txtai/litellm imports are lazy so the package imports in slim environments.
|
||||
|
||||
### Fixed
|
||||
- ISBN regex no longer false-positives on years, URLs, job IDs, or dilutions. Now requires an explicit `ISBN` prefix and matches strict 13- or 10-digit structures.
|
||||
@@ -33,6 +39,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- `import ocr_pipeline` no longer depends on the caller's cwd.
|
||||
|
||||
### Removed
|
||||
- Dead `src/ocr_pipeline/watch.py` shim (shadowed by the `watch/` package).
|
||||
- Unused core dependencies: `tqdm`, `scikit-image`, `sqlite-utils`, `xxhash`, `python-slugify`, `python-magic`, `legacy-cgi`.
|
||||
- Core dependency on `spacy` (moved to `scientific` extra).
|
||||
- Core dependency on `paddleocr`/`paddlepaddle` (moved to `paddle` extra).
|
||||
|
||||
Reference in New Issue
Block a user