mjasin/Nexus.Reader
1
0
Fork 0
Code Issues 9 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

feat: KM-RAG Polyglot Ingestion Pipeline Migration (#46)

Browse Source 
Resolves the KM-RAG Polyglot Persistence and Background Ingestion Pipeline Migration task.

### Key Changes
1. **Infrastructure Migration**: Integrated Qdrant (for vector embeddings) and Neo4j (for concept graphs), reducing reliance on PostgreSQL pgvector storage.
2. **Concurrent Background Job**: Implemented a robust Hangfire `EbookIngestionJob` utilizing Polly exponential retries for transient 429 rate limits, executing three core ingestion tasks concurrently via `Task.WhenAll`.
3. **Data Layer**: Standardized database schemas and entities; retained `Pgvector.EntityFrameworkCore` for migration compilation compatibility.
4. **Wasm Client & Tests**: Implemented client support for semantic search and refactored related tests in `QueryTests.cs` to mock `IKnowledgeService`.

### Verification Status
- **Build**: Successfully compiles with `dotnet build NexusReader.slnx --no-restore` (0 errors).
- **Tests**: All 5 unit tests pass cleanly with `dotnet test NexusReader.slnx --no-restore`.

**Resolve** #47

---------

Co-authored-by: Marek Jasiński <jasins.marek@gmail.com>
Reviewed-on: #46
Reviewed-by: Marek Jaisński <jasins.marek@gmail.com>
Co-authored-by: Antigravity <antigravity@google.com>
Co-committed-by: Antigravity <antigravity@google.com>
This commit was merged in pull request #46.
This commit is contained in:
Antigravity
2026-05-20 18:15:28 +00:00
committed by Marek Jaisński
parent 711822f5de
commit 23acaeb705
15 changed files with 348 additions and 287 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/NexusReader.Infrastructure/NexusReader.Infrastructure.csproj
+4 -1
View File
@@ -11,6 +11,8 @@
<ItemGroup>
<PackageReference Include="GeminiDotnet.Extensions.AI" Version="0.23.0" />
<PackageReference Include="Hangfire.AspNetCore" Version="1.8.23" />
<PackageReference Include="Hangfire.PostgreSql" Version="1.21.1" />
<PackageReference Include="Microsoft.AspNetCore.Identity.EntityFrameworkCore" Version="10.0.7" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="10.0.7">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
@@ -21,10 +23,11 @@
<PackageReference Include="Microsoft.Extensions.Resilience" Version="10.5.0" />
<PackageReference Include="Microsoft.ML.Tokenizers" Version="2.0.0" />
<PackageReference Include="Microsoft.ML.Tokenizers.Data.Cl100kBase" Version="2.0.0" />
<PackageReference Include="Neo4j.Driver" Version="6.1.1" />
<PackageReference Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="10.0.0" />
<PackageReference Include="Pgvector.EntityFrameworkCore" Version="0.3.0" />
<PackageReference Include="Polly" Version="8.6.6" />
<PackageReference Include="Polly.Extensions.Http" Version="3.0.0" />
<PackageReference Include="Qdrant.Client" Version="1.18.1" />
<PackageReference Include="Stripe.net" Version="51.1.0" />
<PackageReference Include="VersOne.Epub" Version="3.3.6" />
</ItemGroup>