CRITICAL: Fix database upsert bug preventing manual scan date persistence
**Root Cause**: upsert_movie_dates() was UPDATE-only, not proper upsert - Manual scans failed to save dateadded to database (NULL values) - Webhooks found database entries but with NULL dateadded - Fell back to current timestamp instead of using proper import dates **Database Fix**: - Changed core/database.py upsert_movie_dates() from UPDATE to INSERT OR REPLACE - Now properly saves dateadded during manual scans - Preserves existing path with COALESCE fallback **Webhook Enhancement**: - Added comprehensive debug logging for database lookups - Enhanced webhook date decision logic with proper fallback chain - Only uses current timestamp as absolute last resort **Impact**: - Movies: Manual scans now persist dates, webhooks find existing entries ✅ - TV Shows: Not affected - already using proper INSERT OR REPLACE ✅ - Version: 1.5.5
This commit is contained in:
+72
-2
@@ -1,8 +1,21 @@
|
|||||||
# NFOGuard Development Summary
|
# NFOGuard Development Summary
|
||||||
|
|
||||||
## Current Status: v1.5.3 - Radarr Rename Event Fix! 🔧
|
## Current Status: v1.5.5 - Database Upsert Fix! 🔧
|
||||||
|
|
||||||
### Latest Updates (v1.5.3) - September 15, 2025
|
### Latest Updates (v1.5.5) - September 15, 2025
|
||||||
|
- **🔧 DATABASE BUG FIX**: Fixed `upsert_movie_dates` to be a proper upsert operation
|
||||||
|
- **💾 DATA INTEGRITY**: Manual scans now properly save `dateadded` to database
|
||||||
|
- **🎯 ROOT CAUSE**: Fixed UPDATE-only operation that was leaving `dateadded` fields NULL
|
||||||
|
- **✅ WEBHOOK FIX**: Webhooks now find existing database entries correctly
|
||||||
|
- **📺 TV SHOWS**: NOT affected - already using proper `INSERT OR REPLACE` operations
|
||||||
|
|
||||||
|
### Previous Updates (v1.5.4) - September 15, 2025
|
||||||
|
- **🎯 WEBHOOK DATE FIX**: Fixed webhook processing to use proper date decision logic
|
||||||
|
- **🔍 SMART FALLBACK**: Webhooks now check database → Radarr import dates → release dates → timestamp
|
||||||
|
- **❌ ISSUE RESOLVED**: Upgrade webhooks no longer incorrectly use current timestamp when database exists
|
||||||
|
- **✅ PROPER FLOW**: Webhook mode now uses same date logic as manual scans for missing entries
|
||||||
|
|
||||||
|
### Previous Updates (v1.5.3) - September 15, 2025
|
||||||
- **🔧 RADARR RENAME FIX**: Added missing event type filtering to Radarr webhooks
|
- **🔧 RADARR RENAME FIX**: Added missing event type filtering to Radarr webhooks
|
||||||
- **✅ CONSISTENCY**: Radarr now matches Sonarr's event handling (Download, Upgrade, Rename)
|
- **✅ CONSISTENCY**: Radarr now matches Sonarr's event handling (Download, Upgrade, Rename)
|
||||||
- **🎯 FIXED ISSUE**: Radarr rename events now properly trigger NFO updates with correct dates
|
- **🎯 FIXED ISSUE**: Radarr rename events now properly trigger NFO updates with correct dates
|
||||||
@@ -63,6 +76,63 @@ SUPPRESS_TVDB_WARNINGS=true # Hide non-critical TVDB failures
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## 🔧 FIXED: Database Upsert Bug (September 15, 2025) - **CRITICAL**
|
||||||
|
|
||||||
|
### Issue Discovery
|
||||||
|
**The Real Root Cause**: Manual scans were NOT actually saving `dateadded` to database due to faulty upsert operation!
|
||||||
|
|
||||||
|
### Problem Details
|
||||||
|
**Database Method Bug** in `core/database.py:173-182`:
|
||||||
|
- `upsert_movie_dates()` was doing UPDATE-only, not proper upsert
|
||||||
|
- If movie row didn't exist properly, UPDATE would affect 0 rows
|
||||||
|
- `dateadded` field remained NULL even after manual scan
|
||||||
|
- Webhooks found database entries but with NULL `dateadded` → fell back to current timestamp
|
||||||
|
|
||||||
|
### 🔧 **Fix Applied**
|
||||||
|
- **Location**: `core/database.py:173-186`
|
||||||
|
- **Changed**: UPDATE-only → proper `INSERT OR REPLACE` upsert
|
||||||
|
- **Result**: Manual scans now properly save `dateadded` to database
|
||||||
|
- **Webhook Impact**: Now finds existing dates correctly, no more false timestamps
|
||||||
|
|
||||||
|
### Before vs After Database Operations
|
||||||
|
**Before (BROKEN):**
|
||||||
|
```sql
|
||||||
|
UPDATE movies SET dateadded = '2025-06-15...' WHERE imdb_id = 'tt123';
|
||||||
|
-- If row doesn't exist properly: 0 rows affected, dateadded stays NULL
|
||||||
|
```
|
||||||
|
|
||||||
|
**After (FIXED):**
|
||||||
|
```sql
|
||||||
|
INSERT OR REPLACE INTO movies (...) VALUES (...);
|
||||||
|
-- Always works: Creates row OR replaces with all data including dateadded
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 FIXED: Webhook Date Logic Issue (September 15, 2025)
|
||||||
|
|
||||||
|
### Issue Discovery
|
||||||
|
Upgrade webhooks were incorrectly using current timestamp (`webhook:first_seen`) instead of checking Radarr for proper import dates when no database entry existed.
|
||||||
|
|
||||||
|
**Example Problem:**
|
||||||
|
- Database populated via manual scan: `2025-06-15T11:10:31-04:00` (radarr:db.history.import)
|
||||||
|
- Upgrade webhook comes in: `2025-09-15T12:04:23-04:00` (webhook:first_seen) ❌ **WRONG!**
|
||||||
|
|
||||||
|
### Root Cause
|
||||||
|
Webhook mode was bypassing the full date decision logic (`_decide_movie_dates`) and jumping straight to current timestamp as fallback.
|
||||||
|
|
||||||
|
### ✅ **Fix Applied**
|
||||||
|
- **Location**: `nfoguard.py:975-990`
|
||||||
|
- **Logic**: Webhook mode now uses same date priority as manual scans:
|
||||||
|
1. **Database first** (if exists)
|
||||||
|
2. **Full date logic** (Radarr import → release dates → fallbacks)
|
||||||
|
3. **Current timestamp** (only as absolute last resort)
|
||||||
|
|
||||||
|
### Result
|
||||||
|
Webhook upgrades now properly find and use existing Radarr import dates instead of creating false "first seen" timestamps.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## 🔧 FIXED: Radarr Rename Event Issue (September 15, 2025)
|
## 🔧 FIXED: Radarr Rename Event Issue (September 15, 2025)
|
||||||
|
|
||||||
### Issue Resolution
|
### Issue Resolution
|
||||||
|
|||||||
+8
-4
@@ -175,11 +175,15 @@ class NFOGuardDatabase:
|
|||||||
"""Insert or update movie date record"""
|
"""Insert or update movie date record"""
|
||||||
with self.get_connection() as conn:
|
with self.get_connection() as conn:
|
||||||
cursor = conn.cursor()
|
cursor = conn.cursor()
|
||||||
|
# Use INSERT OR REPLACE to ensure we always update the dates properly
|
||||||
cursor.execute("""
|
cursor.execute("""
|
||||||
UPDATE movies
|
INSERT OR REPLACE INTO movies (imdb_id, path, released, dateadded, source, has_video_file, last_updated)
|
||||||
SET released = ?, dateadded = ?, source = ?, has_video_file = ?, last_updated = ?
|
VALUES (
|
||||||
WHERE imdb_id = ?
|
?,
|
||||||
""", (released, dateadded, source, has_video_file, datetime.utcnow().isoformat(), imdb_id))
|
COALESCE((SELECT path FROM movies WHERE imdb_id = ?), 'unknown'),
|
||||||
|
?, ?, ?, ?, ?
|
||||||
|
)
|
||||||
|
""", (imdb_id, imdb_id, released, dateadded, source, has_video_file, datetime.utcnow().isoformat()))
|
||||||
|
|
||||||
def get_series_episodes(self, imdb_id: str, has_video_file_only: bool = False) -> List[Dict]:
|
def get_series_episodes(self, imdb_id: str, has_video_file_only: bool = False) -> List[Dict]:
|
||||||
"""Get all episodes for a series"""
|
"""Get all episodes for a series"""
|
||||||
|
|||||||
+17
-13
@@ -970,26 +970,30 @@ class MovieProcessor:
|
|||||||
|
|
||||||
# Get existing dates
|
# Get existing dates
|
||||||
existing = self.db.get_movie_dates(imdb_id)
|
existing = self.db.get_movie_dates(imdb_id)
|
||||||
|
_log("DEBUG", f"Database lookup for {imdb_id}: {existing}")
|
||||||
|
|
||||||
# Handle webhook mode - prioritize database, then use current timestamp
|
# Handle webhook mode - prioritize database, then use proper date logic
|
||||||
if webhook_mode:
|
if webhook_mode:
|
||||||
|
_log("DEBUG", f"Webhook mode: existing={bool(existing)}, has_dateadded={bool(existing and existing.get('dateadded')) if existing else 'N/A'}")
|
||||||
if existing and existing.get("dateadded"):
|
if existing and existing.get("dateadded"):
|
||||||
_log("INFO", f"Webhook processing - using existing database entry: {existing['dateadded']} (source: {existing.get('source', 'unknown')})")
|
_log("INFO", f"Webhook processing - using existing database entry: {existing['dateadded']} (source: {existing.get('source', 'unknown')})")
|
||||||
dateadded, source, released = existing["dateadded"], existing["source"], existing.get("released")
|
dateadded, source, released = existing["dateadded"], existing["source"], existing.get("released")
|
||||||
else:
|
else:
|
||||||
# Use current timestamp as source of truth for webhooks
|
if existing:
|
||||||
local_tz = _get_local_timezone()
|
_log("INFO", f"Webhook processing - database entry exists but no dateadded field: {existing}")
|
||||||
current_time = datetime.now(local_tz).isoformat(timespec="seconds")
|
else:
|
||||||
_log("INFO", f"Webhook processing - using current timestamp as source of truth: {current_time}")
|
_log("INFO", f"Webhook processing - no database entry found for {imdb_id}")
|
||||||
|
_log("INFO", f"Using full date decision logic")
|
||||||
|
# Use same logic as manual scan to check Radarr import dates, release dates, etc.
|
||||||
|
should_query = True # Always query for webhooks when no database entry exists
|
||||||
|
dateadded, source, released = self._decide_movie_dates(imdb_id, movie_path, should_query, existing)
|
||||||
|
|
||||||
# Still get release date for reference
|
# Only if ALL date sources fail, fall back to current timestamp
|
||||||
released = None
|
if dateadded is None:
|
||||||
if config.movie_poll_mode != "never":
|
local_tz = _get_local_timezone()
|
||||||
radarr_movie = self.radarr.movie_by_imdb(imdb_id) if self.radarr.api_key else None
|
current_time = datetime.now(local_tz).isoformat(timespec="seconds")
|
||||||
if radarr_movie:
|
_log("INFO", f"Webhook processing - all date sources failed, using current timestamp as last resort: {current_time}")
|
||||||
released = self._parse_date_to_iso(radarr_movie.get("inCinemas"))
|
dateadded, source = current_time, "webhook:fallback_timestamp"
|
||||||
|
|
||||||
dateadded, source = current_time, "webhook:first_seen"
|
|
||||||
else:
|
else:
|
||||||
# Manual scan mode - determine if we should query APIs
|
# Manual scan mode - determine if we should query APIs
|
||||||
should_query = (
|
should_query = (
|
||||||
|
|||||||
Reference in New Issue
Block a user