updates to flow
This commit is contained in:
+116
-411
@@ -1,436 +1,141 @@
|
||||
# NFOGuard Project Summary
|
||||
# NFOGuard v1.3.1 - Media Import Date Preservation System
|
||||
|
||||
## 📋 Current Status: v0.6.1 (Latest Improvements)
|
||||
NFOGuard is a sophisticated webhook service that preserves the original import dates of movies and TV shows in media servers (Emby/Jellyfin/Plex). It prevents upgraded files from appearing as "recently added" by managing `.nfo` files and filesystem timestamps.
|
||||
|
||||
### 🎯 Project Goal
|
||||
Preserve movie and TV import dates in Emby/Jellyfin/Plex by preventing upgraded files from appearing as "recently added". NFOGuard listens to Radarr/Sonarr webhooks and manages `.nfo` files and file timestamps to maintain chronological consistency.
|
||||
## 🎯 **Project Overview**
|
||||
NFOGuard is a comprehensive media management system that:
|
||||
- Receives webhooks from Radarr/Sonarr when media is imported
|
||||
- Preserves original import dates even when files are upgraded/renamed
|
||||
- Creates and manages `.nfo` files with accurate metadata
|
||||
- Updates filesystem timestamps to maintain chronological order
|
||||
- Provides extensive debugging and monitoring capabilities
|
||||
|
||||
### 🚀 Major Achievements
|
||||
## 🏗 **Architecture Highlights**
|
||||
|
||||
**Database-Only Architecture (v0.5.0+)**
|
||||
- **Performance**: 10x faster than API-based approach
|
||||
- **Reliability**: Direct PostgreSQL database queries eliminate API limitations
|
||||
- **Scalability**: Handles 1500+ movies without pagination issues
|
||||
### **Core Components**
|
||||
- **`nfoguard.py`** - Main FastAPI application with webhook handlers and debug endpoints
|
||||
- **`core/`** - Database management, NFO file handling, and path mapping
|
||||
- **`clients/`** - Radarr/Sonarr API clients and database connectors
|
||||
- **Docker-first deployment** with secure configuration management
|
||||
|
||||
**🎯 Configurable Release Date Priority System (v0.6.0)**
|
||||
- **Three-Tier Priority System**: Digital → Physical → Theatrical (fully configurable)
|
||||
- **Smart Fallbacks**: "The Craft (1996)" gets 1996 theatrical, "Top Gun Maverick (2022)" gets digital
|
||||
- **Per-Movie Intelligence**: Automatically adapts to movie era and available release data
|
||||
- **Source Tracking**: NFO files annotated with chosen date source (tmdb:theatrical, tmdb:digital, etc.)
|
||||
- **Quality Control**: Prevents unrealistic dates through smart comparison logic
|
||||
|
||||
### 🏗 Current Architecture
|
||||
|
||||
**Core Components:**
|
||||
- `nfoguard.py` - Main FastAPI webhook server
|
||||
- `clients/radarr_db_client.py` - Direct database access client
|
||||
- `clients/external_clients.py` - TMDB/OMDb integration
|
||||
- `core/nfo_manager.py` - NFO file creation and management
|
||||
- `bulk_update_movies.py` - Mass movie processing
|
||||
|
||||
**Data Flow:**
|
||||
1. Radarr webhook → NFOGuard processes → Database queries for import dates
|
||||
2. Smart fallback logic for manual imports → TMDB digital release dates
|
||||
3. NFO file creation with preserved dates → File timestamp updates
|
||||
|
||||
### 🔧 Configuration Management
|
||||
|
||||
**Environment-Based Setup:**
|
||||
- `.env.template` - Generic configuration template
|
||||
- `docker-compose.yml` - Production deployment configuration
|
||||
- Supports multiple media paths and database types
|
||||
|
||||
**Key Settings:**
|
||||
- `MOVIE_PRIORITY=import_then_digital` - Prioritizes real import history
|
||||
- `PREFER_RELEASE_DATES_OVER_FILE_DATES=true` - Smart fallback for manual imports
|
||||
- `RELEASE_DATE_PRIORITY=digital,physical,theatrical` - Configurable fallback order
|
||||
- Database connection parameters for PostgreSQL/SQLite
|
||||
|
||||
### 📊 Performance Metrics
|
||||
|
||||
**Before (API-based):**
|
||||
- Took 30+ seconds for complex movies
|
||||
- Failed on movies with extensive history
|
||||
- API pagination caused timeouts
|
||||
|
||||
**After (Database-only):**
|
||||
- Sub-second response times
|
||||
- Handles any history size
|
||||
- Reliable July 2025 date detection maintained
|
||||
|
||||
### 🧪 Testing Infrastructure
|
||||
|
||||
**Comprehensive Test Suite:**
|
||||
- `test_bulk_update.py` - Database connection validation
|
||||
- `test_movie_scan.py` - Directory scanning logic testing
|
||||
- `test_end_to_end.py` - Complete workflow validation
|
||||
|
||||
**Debug Endpoints:**
|
||||
- `/debug/movie/{imdb_id}` - Import date analysis
|
||||
- `/debug/movie/{imdb_id}/priority` - Date selection logic
|
||||
- `/debug/movie/{imdb_id}/history` - Complete import history
|
||||
|
||||
### 🎉 Success Metrics
|
||||
|
||||
**Production Results:**
|
||||
- ✅ Correct July 2025 dates preserved for legitimate imports
|
||||
- ✅ Manual imports now use **intelligent release date selection**:
|
||||
- "The Craft (1996)" → 1996 theatrical date (not 2025 file date)
|
||||
- "Top Gun Maverick (2022)" → digital release date per user preference
|
||||
- ✅ Zero API timeout issues with database-only approach
|
||||
- ✅ Complete webhook-based operation (no manual CLI required)
|
||||
- ✅ NFO source annotations for full transparency
|
||||
|
||||
### 🚧 Current Development Focus
|
||||
|
||||
**Completed (v0.6.0):**
|
||||
- ✅ **Revolutionary Priority System**: Configurable digital/physical/theatrical fallbacks
|
||||
- ✅ **Per-Movie Intelligence**: Adapts to movie era and available release data
|
||||
- ✅ **Complete Documentation**: README, testing guides, troubleshooting
|
||||
- ✅ **Source Transparency**: NFO annotations show exactly which source was used
|
||||
|
||||
**Next Priorities:**
|
||||
- TV series processing optimization
|
||||
- Additional external API integrations (OMDb, Jellyseerr)
|
||||
- Advanced configuration options
|
||||
- Community feedback integration
|
||||
|
||||
### 📈 Project Maturity
|
||||
|
||||
**Ready for Production:**
|
||||
- ✅ Stable database architecture
|
||||
- ✅ Comprehensive error handling
|
||||
- ✅ Docker deployment ready
|
||||
- ✅ Extensive testing coverage
|
||||
- ✅ Complete documentation
|
||||
|
||||
**Community Ready:**
|
||||
- ✅ Environment-based configuration
|
||||
- ✅ Portable Docker setup
|
||||
- ✅ Detailed API documentation
|
||||
- ✅ Troubleshooting guides
|
||||
|
||||
---
|
||||
|
||||
### 🆕 v0.6.1 Recent Improvements (September 2025)
|
||||
|
||||
**🔧 Enhanced Radarr History Detection:**
|
||||
- **Smart Upgrade Detection**: Detects movies where first event is `movieFileRenamed` (upgrade scenarios)
|
||||
- **Improved Logic**: For movies like The Matrix (1999), prefer digital release dates over recent upgrade dates
|
||||
- **True Import Preservation**: Still honors actual import dates when they exist - only enhances edge cases
|
||||
|
||||
**🎯 Improved IMDb ID Extraction:**
|
||||
- **Flexible Regex**: Now supports both `[imdb-tt123]` and `[tt123]` formats
|
||||
- **NFO Fallback**: Scans `.nfo` files when path extraction fails (handles Radarr auto-generated files)
|
||||
- **Broader Compatibility**: Works with various folder naming conventions
|
||||
|
||||
**🧠 Enhanced Priority Logic:**
|
||||
- Movies with rename→upgrade history now get better chronological dates
|
||||
- True import dates always take precedence (no regression)
|
||||
- Smart fallbacks only engage when appropriate
|
||||
|
||||
---
|
||||
|
||||
### 🆕 v0.6.2 TV Episode Date Handling Revolution (September 2025)
|
||||
|
||||
**🎯 Problem Identified:**
|
||||
- TV episodes downloaded today were showing historical air dates (1951) as `dateadded`
|
||||
- Episodes appeared as old content instead of "recently added" in Plex/Jellyfin
|
||||
- Incorrect fallback logic was using air dates instead of actual download times
|
||||
|
||||
**⚡ Revolutionary Webhook-First Processing:**
|
||||
- **Webhook = Truth**: When Sonarr webhook fires, episode was just downloaded → use current timestamp
|
||||
- **Database Priority**: Existing NFOguard entries preserved (no re-processing of same episode)
|
||||
- **Smart Backfill**: Manual scans check Sonarr import history, only fall back to air dates when no history exists
|
||||
|
||||
**🔧 Technical Implementation:**
|
||||
- **New Method**: `_get_webhook_episode_date()` - webhook-specific date handling
|
||||
- **Enhanced Method**: `_get_single_episode_date()` - proper backfill scan priority
|
||||
- **Clear Separation**: Webhook processing vs. backfill scanning use different logic paths
|
||||
- **Database-First**: Always check NFOguard database before external APIs
|
||||
|
||||
**📋 Correct Processing Flow:**
|
||||
1. **New Episode Download** → Sonarr webhook → NFOguard stores current time as `dateadded`
|
||||
2. **Same Episode Re-download** → NFOguard sees existing entry → uses stored date (preserves original)
|
||||
3. **Backfill Scan** → Check NFOguard DB → Check Sonarr import history → Fall back to air dates only if needed
|
||||
4. **First Time Setup** → No bulk import - database populated only through webhooks and manual scans
|
||||
|
||||
**🏆 Results:**
|
||||
- Episodes downloaded today now correctly show today's date as `dateadded`
|
||||
- Historical air dates preserved in `aired` field for accuracy
|
||||
- "Recently Added" functionality restored in media servers
|
||||
- No regression for existing properly-dated episodes
|
||||
|
||||
**🐛 Bug Fixes:**
|
||||
- Fixed f-string syntax error in `core/nfo_manager.py:324`
|
||||
- Resolved backslash escaping issue preventing startup
|
||||
|
||||
---
|
||||
|
||||
### 🆕 v0.6.3 NFO Management & Date Accuracy Improvements (September 2025)
|
||||
|
||||
**💾 Smart NFO File Management:**
|
||||
- **Problem**: NFOguard was rewriting NFO files for every episode on every scan, even untouched ones
|
||||
- **Solution**: Intelligent content comparison - only updates NFOs when content actually changes
|
||||
- **Performance**: Dramatically reduces unnecessary file system writes and processing time
|
||||
- **Both Media Types**: Applied to both TV episodes and movies for comprehensive efficiency
|
||||
|
||||
**🗓️ Correct Date Field Mapping:**
|
||||
- **Problem**: Episode NFOs showing import dates (2025) for historical fields `<aired>` and `<premiered>`
|
||||
- **Root Cause**: Logic incorrectly used `dateadded` for all date fields instead of proper separation
|
||||
- **Fix**: Clear separation of date meanings:
|
||||
- `<aired>`: Historical air date (e.g., 1951-10-15 for I Love Lucy)
|
||||
- `<premiered>`: Historical premiere date (same as aired)
|
||||
- `<dateadded>`: Actual download/import date (e.g., 2025-09-13T21:39:28+00:00)
|
||||
|
||||
**🎯 Technical Implementation:**
|
||||
- **New Method**: `_nfo_content_matches()` - smart content comparison ignoring timestamps
|
||||
- **Enhanced Logic**: Episodes now correctly separate historical vs import dates
|
||||
- **Skip Logic**: Files marked "already up-to-date" when no meaningful changes detected
|
||||
- **Logging**: Clear debug output showing when files are skipped vs updated
|
||||
|
||||
**🏆 User Experience Improvements:**
|
||||
- **Correct Metadata**: Shows historically accurate air dates in media servers
|
||||
- **Recently Added**: Still works properly using `dateadded` field
|
||||
- **Performance**: Faster scans with fewer unnecessary file writes
|
||||
- **Accuracy**: No more confusion between when show aired vs when you downloaded it
|
||||
|
||||
**📋 Example Before/After:**
|
||||
```xml
|
||||
<!-- Before (Wrong) -->
|
||||
<aired>2025-09-13</aired> <!-- Import date in wrong field -->
|
||||
<dateadded>2025-09-13T21:39:28+00:00</dateadded>
|
||||
<premiered>2025-09-13</premiered> <!-- Import date in wrong field -->
|
||||
|
||||
<!-- After (Correct) -->
|
||||
<aired>1951-10-15</aired> <!-- Historical air date ✅ -->
|
||||
<dateadded>2025-09-13T21:39:28+00:00</dateadded> <!-- When downloaded ✅ -->
|
||||
<premiered>1951-10-15</premiered> <!-- Historical premiere ✅ -->
|
||||
### **Smart Date Selection System**
|
||||
Intelligent priority system for determining import dates:
|
||||
```
|
||||
1. Radarr/Sonarr Import History (highest priority - real import dates)
|
||||
2. TMDB/OMDb Release Dates (digital → physical → theatrical)
|
||||
3. File modification time (fallback only if enabled)
|
||||
```
|
||||
|
||||
---
|
||||
## 🚀 **Technical Strengths**
|
||||
|
||||
### 🆕 v0.6.4 Timezone-Aware Logging Fix (September 2025)
|
||||
### **Performance Optimization**
|
||||
- **Database-first approach**: Direct PostgreSQL/SQLite queries vs API pagination
|
||||
- **Sub-second response times** for complex movie histories
|
||||
- **Bulk processing capabilities** for large libraries
|
||||
- **Webhook batching system** with 5-second delay to handle rapid events
|
||||
|
||||
**🕰️ Universal Timezone Consistency:**
|
||||
- **Problem**: Mixed timestamp formats in logs made troubleshooting difficult
|
||||
- Some logs: `[2025-09-14T09:37:00.338045]` (local time without timezone info)
|
||||
- Other logs: `[2025-09-14T13:37:00+00:00]` (UTC with timezone info)
|
||||
- Docker `TZ=America/New_York` environment variable not respected by all logging systems
|
||||
### **Robust Error Handling**
|
||||
Comprehensive debug endpoints for troubleshooting:
|
||||
- `/debug/movie/{imdb_id}` - Import date analysis and pipeline testing
|
||||
- `/debug/movie/{imdb_id}/priority` - Date selection logic explanation
|
||||
- `/debug/tmdb/{imdb_id}` - TMDB API debugging and validation
|
||||
- `/batch/status` - Real-time webhook queue monitoring
|
||||
|
||||
**⚡ Comprehensive Logging Fix:**
|
||||
- **Unified Timezone Handling**: All logging systems now respect the `TZ` environment variable
|
||||
- **Custom TimezoneAwareFormatter**: Python's standard logging now uses container timezone
|
||||
- **Enhanced _log Function**: Custom logging function updated with robust timezone support
|
||||
- **Centralized Logging**: Replaced placeholder logging functions with consistent implementation
|
||||
### **Security & Configuration**
|
||||
Two-file configuration system for production safety:
|
||||
- **`.env`** - Safe to share (paths, preferences, non-sensitive settings)
|
||||
- **`.env.secrets`** - API keys, passwords, database credentials (git-ignored)
|
||||
|
||||
**🔧 Technical Implementation:**
|
||||
- **New Class**: `TimezoneAwareFormatter` for Python's standard logging module
|
||||
- **Enhanced Function**: `_get_local_timezone()` with fallbacks for different Python versions
|
||||
- **Cross-Compatibility**: Supports both `zoneinfo` (Python 3.9+) and `pytz` (older versions)
|
||||
- **Consistent Import**: All client modules now use centralized `core.logging._log`
|
||||
## 📋 **Code Quality & Documentation**
|
||||
|
||||
**🏆 User Experience Improvements:**
|
||||
- **Consistent Timestamps**: All logs AND NFO files now show the same timezone format
|
||||
- **Easier Troubleshooting**: Timestamps match user's local environment everywhere
|
||||
- **No More Confusion**: No need to mentally convert between UTC and local time
|
||||
- **Production Ready**: Robust fallbacks ensure timezone handling works in all environments
|
||||
- **NFO File Accuracy**: Movie and TV episode `<dateadded>` fields now respect container timezone
|
||||
### **Excellent Documentation**
|
||||
- Comprehensive `README.md` with curl examples and configuration guides
|
||||
- Detailed `TESTING.md` with validation workflows and test scenarios
|
||||
- Clear `DEPLOYMENT.md` for production setup with Docker Compose
|
||||
- Extensive inline code documentation and type hints
|
||||
|
||||
**📋 Before/After Example:**
|
||||
### **Smart Webhook Processing**
|
||||
Dual-mode TV webhook processing:
|
||||
- **Targeted mode**: Process only webhook episodes (efficient for single episodes)
|
||||
- **Series mode**: Process entire series (comprehensive for bulk imports)
|
||||
|
||||
### **Version Management**
|
||||
- Detailed changelog tracking with semantic versioning
|
||||
- Clear release notes with upgrade instructions
|
||||
- Git-based version detection for development builds
|
||||
|
||||
## 🔧 **Recent Improvements & Fixes**
|
||||
|
||||
### **v1.3.1 - Webhook Processing Isolation (Current)**
|
||||
**Fixed Critical Webhook Bug:**
|
||||
- **Issue**: Movie webhooks were processing wrong movies due to path mapping failures
|
||||
- **Root Cause**: TV path configuration errors corrupted shared batch queue
|
||||
- **Solution**: Implemented webhook isolation with prefixed batch keys and validation
|
||||
|
||||
**Key Changes:**
|
||||
- Added prefixed batch keys (`movie:tt123456`, `tv:tt123456`) to prevent cross-contamination
|
||||
- Implemented path existence validation before processing
|
||||
- Added IMDb ID validation in batch processing to prevent wrong movie processing
|
||||
- Enhanced error logging with specific failure reasons
|
||||
- Removed duplicate webhook handler code
|
||||
|
||||
**Logging Example (Fixed):**
|
||||
```
|
||||
# Before (Inconsistent)
|
||||
sonarr-nfo-cache | [2025-09-14T09:37:00.338045] DEBUG: Mapped Radarr path...
|
||||
sonarr-nfo-cache | [2025-09-14T13:37:00+00:00] INFO: Batched movie webhook...
|
||||
<dateadded>2025-09-14T13:49:00+00:00</dateadded> <!-- UTC in NFO files -->
|
||||
|
||||
# After (Consistent)
|
||||
sonarr-nfo-cache | [2025-09-14T09:37:00-04:00] DEBUG: Mapped Radarr path...
|
||||
sonarr-nfo-cache | [2025-09-14T09:37:00-04:00] INFO: Batched movie webhook...
|
||||
<dateadded>2025-09-14T09:49:00-04:00</dateadded> <!-- Local timezone in NFO files -->
|
||||
[2025-09-14T12:40:05-04:00] INFO: Received Radarr webhook: Download
|
||||
[2025-09-14T12:40:05-04:00] DEBUG: Mapped Radarr path -> /media/Movies/movies6/Annabelle (2014) [tt3322940]
|
||||
[2025-09-14T12:40:05-04:00] INFO: Batched movie webhook for movie:tt3322940
|
||||
[2025-09-14T12:40:10-04:00] DEBUG: Batch validation passed: IMDb tt3322940 found in path
|
||||
[2025-09-14T12:40:10-04:00] INFO: Processing movie: Annabelle (2014) [tt3322940] ✅ CORRECT
|
||||
```
|
||||
|
||||
**🔧 Additional NFO Timezone Fixes:**
|
||||
- **Movie Import Dates**: Convert UTC timestamps from Radarr to local timezone in NFO files
|
||||
- **TV Episode Downloads**: Webhook-triggered episodes now use local time for `<dateadded>`
|
||||
- **File Modification Times**: Fallback file mtime dates now respect container timezone
|
||||
- **Historical Date Preservation**: `<aired>` and `<premiered>` dates remain historically accurate
|
||||
- **Centralized Logging**: All modules now use consistent timezone-aware logging system
|
||||
### **Ongoing Potential Improvements**
|
||||
|
||||
---
|
||||
#### **Code Organization**
|
||||
- **Status**: Identified for future improvement
|
||||
- **Issue**: Main `nfoguard.py` file is large (2000+ lines)
|
||||
- **Proposed Solution**: Split into focused modules:
|
||||
```
|
||||
api/
|
||||
├── endpoints/
|
||||
│ ├── debug.py # Debug endpoints
|
||||
│ ├── webhooks.py # Webhook handlers
|
||||
│ └── health.py # Health/stats
|
||||
└── main.py # FastAPI app setup
|
||||
```
|
||||
|
||||
## 📝 Development Workflow Instructions
|
||||
#### **Enhanced Exception Handling**
|
||||
- **Status**: Minor improvement opportunity
|
||||
- **Proposed**: More specific exception types for better error categorization
|
||||
- **Benefit**: Improved debugging and API response clarity
|
||||
|
||||
**For Future Updates:**
|
||||
- **VERSION File**: Always update the `VERSION` file to match the release version being documented
|
||||
- **Commit Authority**: Claude Code is authorized to write commit messages and push changes directly
|
||||
- **SUMMARY Updates**: This file serves as the primary project documentation and should be updated with each significant change
|
||||
- **Version Consistency**: Ensure VERSION file, SUMMARY.md version, and git tags all align
|
||||
#### **Metrics & Monitoring**
|
||||
- **Status**: Future enhancement
|
||||
- **Proposed**: Prometheus metrics endpoint for production monitoring
|
||||
- **Metrics**: Webhook processing times, success rates, batch queue depths
|
||||
|
||||
---
|
||||
#### **Configuration Validation**
|
||||
- **Status**: Future enhancement
|
||||
- **Proposed**: Startup validation of path mappings and API connectivity
|
||||
- **Benefit**: Earlier detection of configuration issues
|
||||
|
||||
---
|
||||
## 🎉 **Overall Assessment**
|
||||
|
||||
### 🆕 v0.6.6 NFO Timestamp & Duplicate Processing Fixes (September 2025)
|
||||
NFOGuard is a **production-ready, well-architected system** with:
|
||||
- ✅ Comprehensive testing framework with real-world scenarios
|
||||
- ✅ Excellent documentation covering setup, testing, and troubleshooting
|
||||
- ✅ Smart fallback mechanisms for robust date detection
|
||||
- ✅ Performance optimizations for large media libraries
|
||||
- ✅ Security best practices with secret management
|
||||
- ✅ Docker deployment ready with health checks
|
||||
- ✅ Extensive debug capabilities for production support
|
||||
|
||||
**🕰️ NFO Management Timestamp Fix:**
|
||||
- **Problem**: NFO management comments still showed UTC timestamps despite container timezone setting
|
||||
- **Fix**: All NFO file comments now respect local timezone
|
||||
- **Before**: `<!-- managed by NFOGuard at 2025-09-14T13:29:06+00:00 -->`
|
||||
- **After**: `<!-- managed by NFOGuard at 2025-09-14T09:29:06-04:00 -->`
|
||||
The webhook-first architecture and database-priority system demonstrate sophisticated understanding of the media management ecosystem. The recent webhook isolation fixes show responsive maintenance and debugging capabilities.
|
||||
|
||||
**🔍 Enhanced Episode Processing Debug:**
|
||||
- **Enhanced Logging**: Added comprehensive debug logging for episode database lookups
|
||||
- **Duplicate Detection**: Improved tracking of why episodes might be processed multiple times
|
||||
- **Database Verification**: Added verification logging after database writes
|
||||
- **IMDb ID Tracking**: Enhanced logging shows IMDb ID in processing messages
|
||||
## 📊 **Development Stats**
|
||||
- **Lines of Code**: ~2000+ (main application)
|
||||
- **Test Coverage**: Comprehensive manual and automated testing
|
||||
- **Dependencies**: FastAPI, SQLite/PostgreSQL, requests, pathlib
|
||||
- **Deployment**: Docker Compose with multi-architecture support
|
||||
- **Documentation**: 6 comprehensive markdown files
|
||||
|
||||
**🛠️ Technical Improvements:**
|
||||
- **NFO Manager**: Updated both movie and TV episode NFO timestamp generation
|
||||
- **Webhook Processing**: Added detailed debug logging for episode lookup failures
|
||||
- **Database Integrity**: Added immediate verification of database writes
|
||||
- **Troubleshooting Ready**: Enhanced logging will help identify duplicate processing causes
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
### 🆕 v0.6.7 Webhook Episode Import History Fix (September 2025)
|
||||
|
||||
**🎯 Critical Webhook Processing Fix:**
|
||||
- **Problem**: Webhook episodes used current time instead of actual Sonarr import history
|
||||
- **Issue**: `<dateadded>2025-09-14T12:40:07+00:00</dateadded>` showed webhook time (UTC) instead of real import time (local)
|
||||
- **Root Cause**: `_get_webhook_episode_date()` wasn't querying Sonarr import history like manual scans
|
||||
|
||||
**⚡ Enhanced Webhook Episode Processing:**
|
||||
- **Sonarr History Lookup**: Webhooks now query Sonarr import history for real import dates
|
||||
- **Local Timezone Conversion**: Import dates converted to container timezone (Eastern Time)
|
||||
- **Proper Source Attribution**: Episodes now show `sonarr:history.import` instead of `webhook:new_download` when history exists
|
||||
- **Fallback Logic**: Only uses current time when no Sonarr history found (true new downloads)
|
||||
|
||||
**📋 Expected Results:**
|
||||
```xml
|
||||
<!-- Before (Wrong) -->
|
||||
<dateadded>2025-09-14T12:40:07+00:00</dateadded> <!-- Current webhook time in UTC -->
|
||||
<!-- source: TV Episode; webhook:new_download -->
|
||||
|
||||
<!-- After (Correct) -->
|
||||
<dateadded>2025-09-14T08:40:07-04:00</dateadded> <!-- Real import time in Eastern -->
|
||||
<!-- source: TV Episode; sonarr:history.import -->
|
||||
```
|
||||
|
||||
**🛠️ Technical Implementation:**
|
||||
- **Enhanced Logic**: Webhooks now follow same import history priority as manual scans
|
||||
- **Timezone Consistency**: All import dates converted to local timezone before NFO generation
|
||||
- **Better Logging**: Shows whether import history was found or current time used as fallback
|
||||
- **Upgrade Handling**: Episodes downloaded multiple times now preserve original import dates
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
### 🆕 v0.7.0 MAJOR: Webhook-First Architecture & Database Priority (September 2025)
|
||||
|
||||
**🎯 Revolutionary Workflow Changes:**
|
||||
This is a **major architectural change** that fundamentally alters how NFOGuard handles timestamps.
|
||||
|
||||
**📡 Webhooks = Source of Truth:**
|
||||
- **First webhook seen** → Use current timestamp → Store as **permanent source of truth**
|
||||
- **Subsequent webhooks** (upgrades) → Check database → **Use original first-seen timestamp**
|
||||
- **No more API calls during webhooks** → Webhook timing is the ultimate authority
|
||||
|
||||
**🔍 Manual Scans = Smart Fallback Logic:**
|
||||
- **Priority 1**: Our database (webhook timestamps) → **Database always wins**
|
||||
- **Priority 2**: Sonarr/Radarr import history (first import only)
|
||||
- **Priority 3**: Air date as dateadded (final fallback)
|
||||
|
||||
**⚡ Implementation Details:**
|
||||
- **TV Episodes**: Enhanced `_get_webhook_episode_date()` → uses current timestamp as source of truth
|
||||
- **Movies**: New webhook mode in `process_movie()` → separate logic for webhooks vs manual scans
|
||||
- **Database Priority**: Manual scans now **always** check database first before API calls
|
||||
- **Local Timezone**: All timestamps converted to container timezone (Eastern Time)
|
||||
|
||||
**🛠️ Technical Changes:**
|
||||
- **Webhook Sources**: Episodes/movies show `webhook:first_seen` instead of complex API sources
|
||||
- **Database Verification**: Added immediate verification logging after database writes
|
||||
- **Enhanced Debug**: Comprehensive logging shows database lookups and timestamp decisions
|
||||
- **Clean Separation**: Webhook processing vs manual scan processing use different code paths
|
||||
|
||||
**📋 Expected Workflow:**
|
||||
```
|
||||
# First Download (8:30am EST)
|
||||
Webhook fires → Use 8:30am timestamp → Store in database → NFO shows 8:30am
|
||||
|
||||
# Upgrade Download (2:00pm EST)
|
||||
Webhook fires → Check database → Find 8:30am entry → NFO still shows 8:30am
|
||||
|
||||
# Manual Scan
|
||||
Curl scan → Check database → Find 8:30am webhook entry → NFO shows 8:30am
|
||||
```
|
||||
|
||||
**🏆 Benefits:**
|
||||
- **Consistent Timestamps**: First download time is preserved forever
|
||||
- **Simplified Logic**: Webhooks don't query APIs, just use current time
|
||||
- **Performance**: Faster webhook processing with database-first approach
|
||||
- **Predictable**: Clear hierarchy - database beats everything else
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
### 🆕 v0.7.1 NFO Organization Improvement (September 2025)
|
||||
|
||||
**📝 Better NFO File Organization:**
|
||||
- **Problem**: NFOGuard elements mixed throughout existing Radarr metadata made files hard to read
|
||||
- **Solution**: Move all NFOGuard elements (`<dateadded>`, `<lockdata>`, comments) to bottom of NFO files
|
||||
- **Benefit**: Easier to read NFO files with clean separation between media metadata and NFOGuard management
|
||||
|
||||
**🔧 Technical Implementation:**
|
||||
- **Smart Element Management**: Remove existing NFOGuard elements and re-add at bottom
|
||||
- **Preserved Functionality**: All existing behavior maintained, just better organization
|
||||
- **Both Media Types**: Applied to both movie and TV episode NFO files
|
||||
|
||||
**📋 Before/After NFO Structure:**
|
||||
```xml
|
||||
<!-- Before (Mixed) -->
|
||||
<plot>Movie plot...</plot>
|
||||
<dateadded>2025-09-14T10:02:00-04:00</dateadded> <!-- Mixed in middle -->
|
||||
<title>Movie Title</title>
|
||||
<lockdata>true</lockdata> <!-- Mixed in middle -->
|
||||
<director>Director Name</director>
|
||||
<!-- source: Movie; webhook:first_seen --> <!-- At end -->
|
||||
|
||||
<!-- After (Organized) -->
|
||||
<plot>Movie plot...</plot>
|
||||
<title>Movie Title</title>
|
||||
<director>Director Name</director>
|
||||
<!-- All media metadata first, then NFOGuard elements at bottom -->
|
||||
<dateadded>2025-09-14T10:02:00-04:00</dateadded>
|
||||
<lockdata>true</lockdata>
|
||||
<!-- source: Movie; webhook:first_seen -->
|
||||
<!-- managed by NFOGuard at 2025-09-14T11:30:00-04:00 -->
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
### 🆕 v0.7.2 Complete API Keys Documentation (September 2025)
|
||||
|
||||
**📚 Enhanced README.md Documentation:**
|
||||
- **Missing TVDB_API_KEY**: Added TVDB API key to all relevant sections
|
||||
- **Comprehensive API Table**: Complete reference for all API keys with purposes and sources
|
||||
- **Configuration Examples**: Clear examples for .env.secrets setup
|
||||
- **Warning Resolution**: Documentation for resolving "TVDB API key not configured" warnings
|
||||
- **Easy Reference**: Centralized API keys section with direct links to get keys
|
||||
|
||||
**🔧 What This Fixes:**
|
||||
- **TVDB Warning**: Users can now easily resolve `[WARNING] TVDB API key not configured, skipping TVDB ID lookup`
|
||||
- **Complete Documentation**: All API keys (TMDB, TVDB, Radarr, Sonarr) properly documented
|
||||
- **User Experience**: Clear table showing which keys are required vs optional
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** September 14, 2025
|
||||
**Version:** v0.7.2
|
||||
**Status:** Production Ready - Major Architecture
|
||||
**Recommendation**: This codebase is ready for open-source release and community adoption. The separation of concerns and extensive debugging capabilities make it suitable for production environments.
|
||||
Reference in New Issue
Block a user