360 lines
16 KiB
Markdown
360 lines
16 KiB
Markdown
# NFOGuard Project Overview
|
|
|
|
## 🎯 Three-Component System Architecture
|
|
|
|
**NFOGuard** is a comprehensive media date management ecosystem consisting of three integrated repositories:
|
|
|
|
### 1. **NFOGuard (Docker Service)** [THIS REPOSITORY]
|
|
- **Purpose**: Main webhook service that listens for Sonarr/Radarr events and manages NFO files
|
|
- **Version**: v1.6.0
|
|
- **Technology**: Python 3, Docker, PostgreSQL integration
|
|
- **Role**: Records import dates, creates/updates NFO files, manages filesystem timestamps
|
|
|
|
### 2. **NFOGuard-Emby-Plugin** (Companion DLL)
|
|
- **Purpose**: Emby Server plugin that reads NFO dateadded values and syncs to Emby's DateCreated field
|
|
- **Version**: v0.6.1
|
|
- **Technology**: C# .NET Framework 4.8, Emby SDK
|
|
- **Role**: Real-time date synchronization within Emby, license validation
|
|
|
|
### 3. **nfoguard-license-server** (License Management)
|
|
- **Purpose**: License validation server with admin interface for plugin access control
|
|
- **Version**: v1.3.1
|
|
- **Technology**: Node.js, Express, SQLite
|
|
- **Role**: 7-day trials, license validation, user management dashboard
|
|
|
|
## Project Purpose
|
|
NFOGuard is an automated NFO file management system that integrates with Radarr and Sonarr to maintain accurate metadata and release dates in media library NFO files. It serves as a webhook-driven middleware that preserves existing metadata while adding clean, standardized date information.
|
|
|
|
## Core Architecture
|
|
|
|
### Technology Stack
|
|
- **Language**: Python 3
|
|
- **Database**: PostgreSQL (direct database access to Radarr/Sonarr)
|
|
- **Deployment**: Docker containers with Docker Compose
|
|
- **API Framework**: Built-in HTTP webhook server
|
|
- **File Processing**: XML parsing with xml.etree.ElementTree
|
|
- **Logging**: Custom logging system with configurable debug levels
|
|
|
|
### Key Components
|
|
1. **nfoguard.py** - Main application with webhook server and processing logic
|
|
- Lines 975-990: Webhook date decision logic (fixed in v1.5.4)
|
|
- Lines 1429, 1479: Event type filtering for Sonarr/Radarr webhooks
|
|
- Lines 366, 555, 607, 813: Episode NFO creation calls
|
|
- Lines 586-592: Episode parsing from filename logic
|
|
2. **core/nfo_manager.py** - NFO file creation, parsing, and management
|
|
- Lines 336-375: `_find_existing_episode_nfo()` - Smart NFO detection (v1.6.0)
|
|
- Lines 377-491: `create_episode_nfo()` - Episode NFO creation with renaming
|
|
- Lines 341: Episode filename pattern `S{season_num:02d}E{episode_num:02d}.nfo`
|
|
- Lines 480-487: NFO renaming logic (removes old files after rename)
|
|
- Lines 85-149: `parse_imdb_from_path_or_nfo()` - Dual identification system
|
|
3. **core/database.py** - Database operations and date management
|
|
- Lines 173-186: `upsert_movie_dates()` - CRITICAL FIX: INSERT OR REPLACE (v1.5.5)
|
|
- Database connection handling for Radarr/Sonarr PostgreSQL
|
|
4. **core/logging.py** - Centralized logging system with `_log()` function
|
|
5. **core/config.py** - Configuration management
|
|
|
|
## Primary Features
|
|
|
|
### Webhook Integration
|
|
- **Radarr Webhooks**: `/webhook/radarr` - Handles movie import, upgrade, rename events
|
|
- **Sonarr Webhooks**: `/webhook/sonarr` - Handles TV episode import, upgrade, rename events
|
|
- **Event Filtering**: Processes Download, Upgrade, Rename, Import events only
|
|
- **Batch Processing**: Configurable delays to handle multiple rapid events
|
|
|
|
### Media Identification (Dual Method System)
|
|
1. **Primary**: Directory names with IMDb IDs in brackets `[tt1234567]` or `[imdb-tt1234567]`
|
|
2. **Fallback**: NFO file parsing for IMDb IDs using XML tags:
|
|
- `<uniqueid type="imdb">tt1234567</uniqueid>`
|
|
- `<imdbid>tt1234567</imdbid>`
|
|
- `<imdb>tt1234567</imdb>`
|
|
|
|
### NFO File Management
|
|
|
|
#### Movies
|
|
- **File**: `movie.nfo` in movie directory
|
|
- **Metadata**: Preserves existing Radarr metadata, adds release dates at bottom
|
|
- **Date Priority**: Digital → Physical → Theatrical release dates
|
|
- **Locking**: Adds `<lockdata>true</lockdata>` to prevent overwrites
|
|
|
|
#### TV Shows
|
|
- **Series File**: `tvshow.nfo` in series root directory
|
|
- **Episode Files**: Standardized `S##E##.nfo` format (e.g., `S01E01.nfo`)
|
|
- **Smart Renaming**: Detects existing non-standard NFO files, extracts metadata, renames to standard format
|
|
- **Metadata**: Preserves existing metadata, adds air dates and import dates
|
|
|
|
### Database Integration
|
|
- **Direct Access**: Connects to Radarr/Sonarr PostgreSQL databases
|
|
- **Date Tracking**: Maintains `dateadded` timestamps for import dates
|
|
- **Upsert Operations**: Proper `INSERT OR REPLACE` for data integrity
|
|
- **History Queries**: Retrieves earliest import dates from application databases
|
|
|
|
## Directory Structure Requirements
|
|
|
|
### Movies
|
|
```
|
|
/movies/
|
|
└── Movie Title (2024) [tt1234567]/
|
|
├── movie.mkv
|
|
└── movie.nfo (created/updated by NFOGuard)
|
|
```
|
|
|
|
### TV Shows
|
|
```
|
|
/tv/
|
|
└── Series Title (2024) [tt1234567]/
|
|
├── Season 01/
|
|
│ ├── Series S01E01.mkv
|
|
│ ├── S01E01.nfo (created/updated by NFOGuard)
|
|
│ └── S01E02.nfo
|
|
├── Season 02/
|
|
└── tvshow.nfo (created/updated by NFOGuard)
|
|
```
|
|
|
|
## Configuration System
|
|
|
|
### Environment Files
|
|
- **`.env`**: Main configuration (paths, behavior, non-sensitive options)
|
|
- **`.env.secrets`**: Sensitive data (API keys, database credentials)
|
|
|
|
### Key Settings
|
|
- **Media Paths**: Container and application path mappings
|
|
- **Release Date Priority**: Configurable date preference order
|
|
- **Debug Modes**: `DEBUG`, `PATH_DEBUG`, `SUPPRESS_TVDB_WARNINGS`
|
|
- **Database Credentials**: PostgreSQL connection details
|
|
- **API Keys**: TMDB, TVDB integration (TVDB optional)
|
|
|
|
## API Endpoints
|
|
|
|
| Endpoint | Method | Purpose |
|
|
|----------|--------|---------|
|
|
| `/health` | GET | Health check and status |
|
|
| `/webhook/radarr` | POST | Radarr webhook handler |
|
|
| `/webhook/sonarr` | POST | Sonarr webhook handler |
|
|
| `/manual/scan` | POST | Manual media scanning |
|
|
| `/bulk/update` | POST | Bulk movie updates from Radarr DB |
|
|
|
|
## Critical Bug Fixes & Technical Details
|
|
|
|
### Database Upsert Bug (v1.5.5) - CRITICAL
|
|
- **Location**: `core/database.py:173-186`
|
|
- **Problem**: `upsert_movie_dates()` was UPDATE-only, not proper upsert
|
|
- **Symptoms**: Manual scans weren't saving `dateadded`, webhooks fell back to current timestamp
|
|
- **Fix**: Changed to `INSERT OR REPLACE` with `COALESCE` for path preservation
|
|
- **Impact**: Fixed webhook date accuracy and database integrity
|
|
|
|
### Webhook Date Logic Bug (v1.5.4)
|
|
- **Location**: `nfoguard.py:975-990`
|
|
- **Problem**: Upgrade webhooks bypassed full date decision logic
|
|
- **Symptoms**: Used current timestamp instead of existing Radarr import dates
|
|
- **Fix**: Webhook mode now uses same date priority as manual scans
|
|
- **Flow**: Database → Radarr/Sonarr API → Release dates → Current timestamp
|
|
|
|
### Radarr Event Filtering (v1.5.3)
|
|
- **Location**: `nfoguard.py:1479`
|
|
- **Problem**: Missing event type filtering (Sonarr had it, Radarr didn't)
|
|
- **Fix**: Added `if event_type not in ["Download", "Upgrade", "Rename"]:`
|
|
- **Result**: Both systems now follow identical webhook processing
|
|
|
|
### NFO-Based Identification System (v1.6.0)
|
|
- **Location**: `core/nfo_manager.py:85-149`
|
|
- **Method**: `parse_imdb_from_path_or_nfo()` tries directory first, then NFO files
|
|
- **Supports**: `<uniqueid type="imdb">`, `<imdbid>`, `<imdb>` XML tags
|
|
- **Files**: movie.nfo and tvshow.nfo parsing
|
|
|
|
### Smart Episode NFO Renaming (v1.6.0)
|
|
- **Location**: `core/nfo_manager.py:336-491`
|
|
- **Detection**: `_find_existing_episode_nfo()` scans season directories
|
|
- **Process**: Extract metadata → Write to standard format → Remove old file
|
|
- **Pattern**: Always creates `S##E##.nfo` format
|
|
- **Safety**: Only removes old file after successful write
|
|
|
|
## Development History & Major Fixes
|
|
|
|
### v1.6.0 (September 16, 2025) - Current
|
|
- **NFO-Based Identification**: Added dual identification system (directory + NFO parsing)
|
|
- **Smart NFO Renaming**: Detects and renames non-standard episode NFO files to `S##E##.nfo` format
|
|
- **Metadata Preservation**: Maintains existing metadata during NFO operations
|
|
- **Enhanced Documentation**: Comprehensive README and project documentation
|
|
|
|
### v1.5.5 (September 15, 2025)
|
|
- **CRITICAL DATABASE FIX**: Fixed `upsert_movie_dates()` from UPDATE-only to proper `INSERT OR REPLACE`
|
|
- **Data Integrity**: Manual scans now properly save `dateadded` to database
|
|
- **Webhook Fix**: Webhooks now find existing database entries correctly
|
|
|
|
### v1.5.4 (September 15, 2025)
|
|
- **Webhook Date Logic Fix**: Fixed webhook processing to use proper date decision logic
|
|
- **Smart Fallback**: Database → Radarr import dates → release dates → timestamp priority
|
|
|
|
### v1.5.3 (September 15, 2025)
|
|
- **Radarr Rename Fix**: Added missing event type filtering to Radarr webhooks
|
|
- **Consistency**: Radarr now matches Sonarr's event handling
|
|
|
|
### v1.5.2
|
|
- **TVDB Warning Suppression**: Added `SUPPRESS_TVDB_WARNINGS` for cleaner logs
|
|
- **Production Ready**: Reduced log noise for production environments
|
|
|
|
## Date Decision Logic
|
|
|
|
### Movies
|
|
1. **Database Check**: Existing NFOGuard database entry
|
|
2. **Radarr History**: Earliest import date from Radarr database
|
|
3. **Release Dates**: Digital → Physical → Theatrical (configurable priority)
|
|
4. **File Dates**: File system timestamps (fallback)
|
|
5. **Current Timestamp**: Absolute last resort
|
|
|
|
### TV Shows
|
|
1. **Database Check**: Existing NFOGuard database entry
|
|
2. **Sonarr History**: Earliest import date from Sonarr database
|
|
3. **Air Dates**: Episode air date from metadata
|
|
4. **File Dates**: File system timestamps (fallback)
|
|
5. **Current Timestamp**: Absolute last resort
|
|
|
|
## Workflow Patterns
|
|
|
|
### Webhook Processing
|
|
1. **Event Validation**: Filter supported event types
|
|
2. **Media Detection**: Parse file paths and identify media
|
|
3. **Database Lookup**: Check for existing entries
|
|
4. **Date Resolution**: Apply decision logic hierarchy
|
|
5. **NFO Processing**: Create/update NFO files with metadata
|
|
6. **Database Update**: Store processed information
|
|
|
|
### Manual Scanning
|
|
1. **Directory Traversal**: Scan configured media paths
|
|
2. **Media Identification**: Directory names and NFO files
|
|
3. **Batch Processing**: Process multiple items efficiently
|
|
4. **Database Persistence**: Store all discovered media
|
|
5. **NFO Creation**: Generate/update all NFO files
|
|
|
|
## Integration Points
|
|
|
|
### External Services
|
|
- **Radarr**: Movie management, webhook triggers, database queries
|
|
- **Sonarr**: TV show management, webhook triggers, database queries
|
|
- **TMDB**: Movie metadata and release dates (primary)
|
|
- **TVDB**: TV show metadata (optional, failures suppressed)
|
|
- **Emby/Jellyfin**: NFO consumption for media servers
|
|
|
|
### File System
|
|
- **Read Access**: Media directories for scanning and identification
|
|
- **Write Access**: NFO file creation and updates
|
|
- **Path Mapping**: Container paths vs application paths
|
|
|
|
## Development Patterns
|
|
|
|
### Code Organization
|
|
- **Modular Design**: Separate concerns (database, NFO, logging, config)
|
|
- **Error Handling**: Comprehensive exception handling with logging
|
|
- **Configuration Driven**: Environment-based configuration management
|
|
- **Docker Native**: Designed for containerized deployment
|
|
|
|
### Testing Strategy
|
|
- **Manual Testing**: API endpoints and webhook simulation
|
|
- **Real-world Integration**: Actual Radarr/Sonarr webhook testing
|
|
- **Database Validation**: Direct database query verification
|
|
- **File System Testing**: NFO file creation and parsing validation
|
|
|
|
## Common Issues & Solutions
|
|
|
|
### Path Mapping
|
|
- **Problem**: Container paths don't match application paths
|
|
- **Solution**: Proper volume mapping and path configuration in `.env`
|
|
- **Debug**: Enable `PATH_DEBUG=true` to see path resolution
|
|
|
|
### Database Connectivity
|
|
- **Problem**: Cannot connect to Radarr/Sonarr databases
|
|
- **Solution**: Verify credentials in `.env.secrets` and network connectivity
|
|
- **Check**: Database host, port, username, password in secrets file
|
|
|
|
### Permission Issues
|
|
- **Problem**: Cannot write NFO files
|
|
- **Solution**: Ensure proper file permissions and volume mount configurations
|
|
- **Fix**: `chmod 755` on media directories, `rw` mount permissions
|
|
|
|
### TVDB Failures
|
|
- **Problem**: TVDB API errors causing log noise
|
|
- **Solution**: Set `SUPPRESS_TVDB_WARNINGS=true` (system works without TVDB)
|
|
- **Note**: TVDB is optional - IMDb and Sonarr data are primary sources
|
|
|
|
### Webhook Issues
|
|
- **Problem**: Webhooks not triggering NFO updates
|
|
- **Debug Steps**:
|
|
1. Check webhook URLs: `http://nfoguard:8080/webhook/radarr|sonarr`
|
|
2. Verify event types: Download, Upgrade, Rename only
|
|
3. Enable `DEBUG=true` for detailed webhook logging
|
|
4. Check database entries with manual scan first
|
|
|
|
### Date Problems
|
|
- **Problem**: Wrong dates in NFO files (current timestamp instead of import date)
|
|
- **Root Cause**: Usually database upsert issues or missing database entries
|
|
- **Solution**: Run manual scan first to populate database, then test webhooks
|
|
- **Priority**: Database → API → Release dates → File dates → Current timestamp
|
|
|
|
## Future Considerations
|
|
|
|
### Potential Enhancements
|
|
- **Movie NFO Renaming**: Similar to TV show episode renaming
|
|
- **Multiple NFO Formats**: Support for different media server preferences
|
|
- **Metadata Enhancement**: Additional metadata sources and fields
|
|
- **Performance Optimization**: Caching and batch processing improvements
|
|
|
|
### Monitoring & Maintenance
|
|
- **Health Checks**: Regular endpoint monitoring
|
|
- **Log Analysis**: Debug mode for troubleshooting
|
|
- **Database Maintenance**: Regular cleanup of old entries
|
|
- **Version Management**: Semantic versioning and change tracking
|
|
|
|
## Development Environment
|
|
|
|
### Prerequisites
|
|
- Docker and Docker Compose
|
|
- Python 3.x development environment
|
|
- Access to Radarr/Sonarr instances with PostgreSQL
|
|
- Git workflow familiarity
|
|
|
|
### Development Workflow
|
|
1. **Local Testing**: Use development Docker image
|
|
2. **Code Changes**: Implement features with proper logging
|
|
3. **Version Updates**: Update VERSION file and SUMMARY.md
|
|
4. **Documentation**: Update README.md and Project.md
|
|
5. **Deployment**: Push to dev branch, then production
|
|
|
|
## Quick Context Recovery
|
|
|
|
### When Starting a New Session
|
|
1. **Read Project.md** - This file for full context
|
|
2. **Check SUMMARY.md** - Latest changes and current version
|
|
3. **Review VERSION** - Current version number (v1.6.0)
|
|
4. **Key Files to Understand**:
|
|
- `nfoguard.py` - Main logic, webhook handlers
|
|
- `core/nfo_manager.py` - NFO creation, parsing, renaming
|
|
- `core/database.py` - Database operations (watch the upsert bug fix!)
|
|
|
|
### Critical Code Locations to Remember
|
|
- **Database Upsert**: `core/database.py:173-186` (fixed UPDATE → INSERT OR REPLACE)
|
|
- **Webhook Date Logic**: `nfoguard.py:975-990` (fixed fallback behavior)
|
|
- **Event Filtering**: `nfoguard.py:1429,1479` (Download/Upgrade/Rename only)
|
|
- **NFO Renaming**: `core/nfo_manager.py:336-491` (smart episode renaming)
|
|
- **Dual Identification**: `core/nfo_manager.py:85-149` (directory + NFO parsing)
|
|
|
|
### Project Context (Updated 2025-09-17)
|
|
- **Current Status**: Production-ready system with 3-component architecture
|
|
- **Version**: v1.6.0 (NFOGuard Docker), v1.1.18 (Emby Plugin), v1.3.2 (License Server)
|
|
- **User Environment**: Docker deployment, PostgreSQL databases, Unmanic integration
|
|
- **Focus Areas**: Webhook accuracy, NFO standardization, date integrity, lifetime license management
|
|
- **Testing Method**: Real webhooks from Radarr/Sonarr, manual scans for validation
|
|
- **Recent Updates**: Fixed lifetime license display issue with explicit "never" expires handling
|
|
|
|
### Three-Repository System Overview
|
|
1. **NFOGuard (Docker)** - Main webhook service and NFO management
|
|
2. **NFOGuard-Emby-Plugin** - DLL for real-time Emby date synchronization
|
|
3. **nfoguard-license-server** - License validation and user management
|
|
|
|
### Recent Work Pattern
|
|
- User reports issues with specific examples (log snippets, webhook behavior)
|
|
- We debug by analyzing code at specific line numbers
|
|
- Implement fixes with proper error handling and logging
|
|
- Update documentation (SUMMARY.md, README.md, Project.md) across all repositories
|
|
- Version bump and prepare for deployment
|
|
|
|
This project represents a mature, production-ready system for automated NFO management with comprehensive webhook integration, intelligent metadata handling, and integrated license management. |