Files
nfoguard/SUMMARY.md
T
sbcrumb 1353b53cc8 CRITICAL: Fix database upsert bug preventing manual scan date persistence
**Root Cause**: upsert_movie_dates() was UPDATE-only, not proper upsert
- Manual scans failed to save dateadded to database (NULL values)
- Webhooks found database entries but with NULL dateadded
- Fell back to current timestamp instead of using proper import dates

**Database Fix**:
- Changed core/database.py upsert_movie_dates() from UPDATE to INSERT OR REPLACE
- Now properly saves dateadded during manual scans
- Preserves existing path with COALESCE fallback

**Webhook Enhancement**:
- Added comprehensive debug logging for database lookups
- Enhanced webhook date decision logic with proper fallback chain
- Only uses current timestamp as absolute last resort

**Impact**:
- Movies: Manual scans now persist dates, webhooks find existing entries 
- TV Shows: Not affected - already using proper INSERT OR REPLACE 
- Version: 1.5.5
2025-09-15 12:18:53 -04:00

8.2 KiB

NFOGuard Development Summary

Current Status: v1.5.5 - Database Upsert Fix! 🔧

Latest Updates (v1.5.5) - September 15, 2025

  • 🔧 DATABASE BUG FIX: Fixed upsert_movie_dates to be a proper upsert operation
  • 💾 DATA INTEGRITY: Manual scans now properly save dateadded to database
  • 🎯 ROOT CAUSE: Fixed UPDATE-only operation that was leaving dateadded fields NULL
  • WEBHOOK FIX: Webhooks now find existing database entries correctly
  • 📺 TV SHOWS: NOT affected - already using proper INSERT OR REPLACE operations

Previous Updates (v1.5.4) - September 15, 2025

  • 🎯 WEBHOOK DATE FIX: Fixed webhook processing to use proper date decision logic
  • 🔍 SMART FALLBACK: Webhooks now check database → Radarr import dates → release dates → timestamp
  • ISSUE RESOLVED: Upgrade webhooks no longer incorrectly use current timestamp when database exists
  • PROPER FLOW: Webhook mode now uses same date logic as manual scans for missing entries

Previous Updates (v1.5.3) - September 15, 2025

  • 🔧 RADARR RENAME FIX: Added missing event type filtering to Radarr webhooks
  • CONSISTENCY: Radarr now matches Sonarr's event handling (Download, Upgrade, Rename)
  • 🎯 FIXED ISSUE: Radarr rename events now properly trigger NFO updates with correct dates
  • 🔄 WORKFLOW: Both Sonarr and Radarr now follow identical webhook processing flows

Previous Updates (v1.5.2)

  • 🔇 TVDB WARNINGS: Added SUPPRESS_TVDB_WARNINGS configuration option
  • CLEAN LOGS: Can now suppress non-critical TVDB API failure warnings
  • 🧹 PRODUCTION: Cleaner logs for production monitoring
  • 🎯 FUNCTIONAL: System works perfectly without TVDB (uses IMDb/Sonarr data)

TVDB Warning Suppression

Added new environment variable:

# Suppress TVDB API warnings (true/false) - TVDB failures are common and non-critical
SUPPRESS_TVDB_WARNINGS=true

Why Suppress TVDB Warnings?

  • Common Failures: Many series don't exist in TVDB database
  • Non-Critical: NFOGuard works perfectly without TVDB data
  • 📊 Primary Data: Uses IMDb IDs and Sonarr metadata (more reliable)
  • 🧹 Log Noise: Reduces repetitive warning messages

Before vs After

Before (noisy):

INFO: All episodes found in cache
WARNING: GET https://api4.thetvdb.com/v4/search/remoteid?remoteId=tt0804484&type=series failed: HTTP Error 400: Bad Request
INFO: Completed processing TV series: Foundation (2021) [imdb-tt0804484]
WARNING: GET https://api4.thetvdb.com/v4/search/remoteid?remoteId=tt9737326&type=series failed: HTTP Error 400: Bad Request

After (clean):

INFO: All episodes found in cache
INFO: Completed processing TV series: Foundation (2021) [imdb-tt0804484]
INFO: Completed processing TV series: Invasion (2021) [imdb-tt9737326]

Your Configuration

DEBUG=false                    # Clean production logging
PATH_DEBUG=false              # No path mapping debug
SUPPRESS_TVDB_WARNINGS=true   # Hide non-critical TVDB failures

System Status

  • Processing: All TV series processing successfully
  • Episodes: All episodes found and processed
  • NFO Files: Being created/updated with proper metadata
  • Functionality: TVDB failures don't affect operation

🔇 NFOGuard v1.5.2 - Ultra-clean production logs! 🔇

Copy SUPPRESS_TVDB_WARNINGS=true to your server .env file and restart!


🔧 FIXED: Database Upsert Bug (September 15, 2025) - CRITICAL

Issue Discovery

The Real Root Cause: Manual scans were NOT actually saving dateadded to database due to faulty upsert operation!

Problem Details

Database Method Bug in core/database.py:173-182:

  • upsert_movie_dates() was doing UPDATE-only, not proper upsert
  • If movie row didn't exist properly, UPDATE would affect 0 rows
  • dateadded field remained NULL even after manual scan
  • Webhooks found database entries but with NULL dateadded → fell back to current timestamp

🔧 Fix Applied

  • Location: core/database.py:173-186
  • Changed: UPDATE-only → proper INSERT OR REPLACE upsert
  • Result: Manual scans now properly save dateadded to database
  • Webhook Impact: Now finds existing dates correctly, no more false timestamps

Before vs After Database Operations

Before (BROKEN):

UPDATE movies SET dateadded = '2025-06-15...' WHERE imdb_id = 'tt123';
-- If row doesn't exist properly: 0 rows affected, dateadded stays NULL

After (FIXED):

INSERT OR REPLACE INTO movies (...) VALUES (...);
-- Always works: Creates row OR replaces with all data including dateadded

🎯 FIXED: Webhook Date Logic Issue (September 15, 2025)

Issue Discovery

Upgrade webhooks were incorrectly using current timestamp (webhook:first_seen) instead of checking Radarr for proper import dates when no database entry existed.

Example Problem:

  • Database populated via manual scan: 2025-06-15T11:10:31-04:00 (radarr:db.history.import)
  • Upgrade webhook comes in: 2025-09-15T12:04:23-04:00 (webhook:first_seen) WRONG!

Root Cause

Webhook mode was bypassing the full date decision logic (_decide_movie_dates) and jumping straight to current timestamp as fallback.

Fix Applied

  • Location: nfoguard.py:975-990
  • Logic: Webhook mode now uses same date priority as manual scans:
    1. Database first (if exists)
    2. Full date logic (Radarr import → release dates → fallbacks)
    3. Current timestamp (only as absolute last resort)

Result

Webhook upgrades now properly find and use existing Radarr import dates instead of creating false "first seen" timestamps.


🔧 FIXED: Radarr Rename Event Issue (September 15, 2025)

Issue Resolution

COMPLETED: Fixed missing Radarr rename event handling by adding proper event type filtering.

Both Sonarr & Radarr Webhook Handling - NOW WORKING CORRECTLY

  • Sonarr Location: nfoguard.py:1429
  • Radarr Location: nfoguard.py:1479 (FIXED)
  • Event Filtering: if event_type not in ["Download", "Upgrade", "Rename"]:
  • STATUS: Rename events ARE processed for both systems
  • BEHAVIOR: Both systems now update NFO files with proper date metadata on rename

What Was Fixed

  1. Added Event Type Filtering: Radarr webhooks now explicitly filter for supported events
  2. Consistent Processing: Both Sonarr and Radarr now follow identical webhook workflows
  3. Proper Rename Handling: Rename events trigger full NFO processing pipeline

📋 NFOGuard Webhook Workflow Documentation

🎬 Radarr Webhook Workflow

  1. Import Event:

    • Check database - if not seen before, current timestamp = truth of first import
    • Update movie.nfo with proper date metadata
  2. Rename Event:

    • Check database - if we have the date, use existing date
    • Update movie.nfo file as always (preserving original import date)
  3. Rename Event (No Database Entry):

    • Check Radarr database for earliest import date
    • If found, use earliest import date → update database + movie.nfo
    • If no import date found, use digital/theatrical release date (fallback logic)

📺 Sonarr Webhook Workflow

  1. Import Event:

    • Check database - if not seen before, current timestamp = truth of first import
    • Update episode.nfo with proper date metadata
  2. Rename Event:

    • Check database - if we have the date, use existing date
    • Update episode.nfo file as always (preserving original import date)
  3. Rename Event (No Database Entry):

    • Check Sonarr API for earliest import date
    • If found, use earliest import date → update database + episode.nfo
    • If no import date found, use air date (fallback logic)

🔄 Key Processing Logic

  • Database First: Always check NFOGuard database for existing dates
  • Import Truth: Webhook import events establish "source of truth" timestamps
  • Date Preservation: Rename events preserve original import dates, never overwrite
  • Smart Fallbacks: API queries → Release dates → Air dates → File dates (configurable)
  • Batch Processing: All webhooks go through batching system with configurable delays