Lossless Memory Manager

Zero-loss memory management: semantic clustering, smart compaction, and cross-session recall. Never forget what matters.

Commands

Run via: python3 scripts/lossless-memory-manager.py [options]

Core Commands

• index — Build/rebuild semantic index of all memory files
• search --query [--top ] — Semantic search across all memory files
• tag --auto — Auto-tag untagged memory entries (categories, importance, entities)
• tag --file --tags — Manually tag a memory file
• compact [--older-than ] — Smart compaction: merge similar entries, preserve unique info
• health — Memory health report: sizes, gaps, duplicates, growth rate
• timeline --days — Show chronological timeline of key events
• entities — Extract and list all named entities (people, projects, tools) across memory

Options

• --json — JSON output
• --verbose — Show detailed processing info
• --dry-run — Show what would change without modifying files

How It Works

Semantic Indexing

The index command builds a keyword-based semantic index of all .md files in memory/:

• Extracts headings, key phrases, dates, named entities
• Builds inverted index for fast retrieval
• Stores index in memory/.memory-index.json

Zero-Loss Compaction

Unlike simple compression, lossless compaction:

1. Identifies duplicate information across daily files

2. Merges similar entries keeping the most detailed version

3. Preserves unique details even from old entries

4. Creates audit trail — original content archived, not deleted

5. Never summarizes away specifics — dates, numbers, names always preserved

Auto-Tagging

The tag --auto command categorizes entries by:

• Importance — critical, important, routine, trivial
• Category — decision, event, learning, todo, context, financial, technical
• Entities — People, projects, tools, companies mentioned

Data Files

• memory/.memory-index.json — Semantic index (rebuilt by index)
• memory/.memory-tags.json — Tag database
• memory/.memory-entities.json — Extracted entities database

Cron Integration

Daily indexing and health check:

lossless-memory-manager.py index && lossless-memory-manager.py health

Weekly compaction:

lossless-memory-manager.py compact --older-than 14

References

• See references/compaction-algorithm.md for the zero-loss compaction algorithm