sbcrumb/nfoguard
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
22483717c45626b245e89e004a8e5f35e27d033d
nfoguard/SUMMARY.md
T
sbcrumb 22483717c4 initial update
2025-09-21 09:25:36 -04:00

22 KiB
Raw Blame History

NFOGuard Project Summary

Overview

NFOGuard is a Python-based tool that manages NFO (metadata) files for movies and TV shows in Plex/Emby media servers. It integrates with Radarr and Sonarr to automatically manage metadata, ensuring proper dates and metadata consistency.

🎯 CURRENT SESSION ACCOMPLISHMENTS (September 21, 2025)

✅ MAJOR ISSUES RESOLVED

  1. Database Save Bug: Identified and resolved "phantom" database save issue (was logging interpretation error)
  2. Enhanced Movie Detection: Movies without IMDb IDs in directory names now detected via filenames/NFO content
  3. Enhanced Logging: Crystal-clear source attribution showing NFOGuard DB vs Radarr vs External APIs
  4. Priority Flow Confirmed: Perfect implementation of user's requested flow

✅ KEY IMPROVEMENTS MADE

🔍 Enhanced Movie Detection:

  • Added find_movie_imdb_id() function with comprehensive detection
  • Supports directory names, filenames, and NFO file content parsing
  • Movies like "Ick (2024)" now properly processed without directory IMDb tags

📊 Crystal-Clear Logging System:

  • Added emoji-based logging for easy visual scanning
  • Clear source attribution: nfoguard_db:, radarr:db., tmdb:, etc.
  • Priority flow display shows exact decision path
  • Final completion logs show chosen date and source

🔄 Perfect Priority Flow:

  1. NFOGuard Database (cached entries) →
  2. Radarr Import History (real import dates) →
  3. External APIs (TMDB/OMDb with configurable priority) →
  4. Save to NFOGuard Database (cache for future)

🚀 Manual Scan Commands:

  • Movies only: curl -X POST "http://nfoguard:8080/manual/scan?scan_type=movies"
  • TV only: curl -X POST "http://nfoguard:8080/manual/scan?scan_type=tv"
  • Both: curl -X POST "http://nfoguard:8080/manual/scan?scan_type=both"

✅ CONFIGURATION MAINTAINED

All existing .env settings fully preserved and respected:

  • RELEASE_DATE_PRIORITY=digital,physical,theatrical
  • MOVIE_PRIORITY=import_then_digital
  • PREFER_RELEASE_DATES_OVER_FILE_DATES=true
  • ALLOW_FILE_DATE_FALLBACK=false

✅ DEBUGGING COMPLETED

  • Root Cause: "Phantom database save" was actually two different movie executions in logs
  • Database Save: Working correctly in both early-exit and full-processing paths
  • NFO Processing: Fixed to execute before date validation (preserves existing metadata)
  • File Detection: Enhanced to handle movies without directory IMDb tags

Core Architecture

Main Components

  • nfoguard.py: Main application file containing the core logic
  • core/nfo_manager.py: Handles NFO file creation and updates
  • clients/: External API integrations (Radarr, Sonarr, etc.)

Key Features

  • Creates and updates movie.nfo files with proper metadata
  • Manages TV show NFO files (tvshow.nfo, season.nfo, episode NFOs)
  • Preserves existing metadata while adding NFOGuard-managed fields
  • Supports date management from multiple sources (Radarr, file dates, etc.)
  • Webhook support for real-time processing

Current Issue Analysis

Problem

When a movie.nfo file already exists (e.g., from Radarr), NFOGuard is not processing the file to:

  1. Move date fields to the bottom
  2. Add NFOGuard comment
  3. Add NFOGuard-managed fields (dateadded, lockdata, etc.)

Root Cause Identified

The issue is in nfoguard.py:1009-1012. The processing logic has a premature exit condition:

# Skip processing if no valid date found and file dates disabled  
if dateadded is None:
    _log("WARNING", f"Skipping movie {movie_path.name} - no valid date source available")
    self.db.upsert_movie_dates(imdb_id, released, None, source, True)
    return  # <-- This exits BEFORE NFO processing

The NFO processing code (lines 1015-1018) never executes when dateadded is None.

Analysis of NFO Processing Logic

The create_movie_nfo function in core/nfo_manager.py:52-147 is actually well-designed:

  • Lines 60-78: Properly loads and preserves existing NFO content
  • Lines 69-78: Removes NFOGuard-managed fields to re-add them at the bottom
  • Lines 96-123: Adds all NFOGuard fields at the end
  • Lines 124-141: Adds NFOGuard comment and proper formatting

The NFO manager code is correct - it's just not being called due to the early exit.

Files That Need Updates

  1. nfoguard.py - Fix the early exit logic around line 1009-1012

Status

  • ✅ NFO Processing Issue: Fixed in nfoguard.py:1008-1025
  • 🔍 NEW ISSUE: Database not persisting dateadded values on manual scans

Fix Summary

Movie NFO Processing (✅ Fixed)

Problem: NFO processing was skipped entirely when dateadded was None due to early return.

Solution: Moved NFO processing (create_movie_nfo) to execute before the dateadded check, ensuring that:

  1. Existing NFO files are always processed and standardized
  2. NFOGuard comment is added
  3. Date fields are moved to bottom
  4. lockdata is added if configured
  5. Only file mtime updates are skipped when no valid date is found

TV Episode NFO Processing (✅ Enhanced)

Enhancement: Added smart episode NFO migration for long-named files.

New Functionality:

  1. Detection: find_existing_episode_nfo() scans for non-standard NFO files that match season/episode
  2. Migration: Extracts metadata from long-named files (e.g., Episode.Name.Here.S01E05.nfo)
  3. Standardization: Creates standard S01E05.nfo with preserved metadata + NFOGuard enhancements
  4. Cleanup: Automatically deletes old long-named NFO files after successful migration
  5. Preservation: All existing metadata is preserved, dates moved to bottom, NFOGuard fields added

🔍 DATABASE SAVE BUG RESOLVED! ✅

Problem Analysis COMPLETED

Root Cause Identified: The "phantom database save bug" was actually a logging interpretation error.

What Actually Happened

The debug logs showing these two lines:

DEBUG: Movie processing reached post-mtime section: Dear Santa (2024) [imdb-tt2396431]
INFO: Completed processing movie: Dear Santa (2024) [imdb-tt2396431] (source: tmdb:digital)

Cannot be from the same function execution! Here's why:

The Real Execution Flow

Line 1014-1018: Early exit when dateadded is None:

if dateadded is None:
    _log("WARNING", f"Movie {movie_path.name} - no valid date source available, but NFO was still processed")
    self.db.upsert_movie_dates(imdb_id, released, None, source, True)
    return  # <-- EXITS HERE, never reaches line 1029!

Line 1029: Debug log that only executes when dateadded has a valid value:

_log("DEBUG", f"Movie processing reached post-mtime section: {movie_path.name}")

Line 1045: Completion log that only executes after successful database save:

_log("INFO", f"Completed processing movie: {movie_path.name} (source: {source})")

Key Insight

The logs are from TWO different executions:

  1. First run: dateadded = None → early exit at line 1018 → database save executes BUT for NULL value
  2. Second run: dateadded has value → reaches line 1029 → full processing → line 1045

Database Save Status

✅ Database saves ARE working correctly!

  • Early exit path (line 1018): Saves NULL dateadded when no valid date found
  • Full processing path (lines 1038-1043): Saves valid dateadded when available
  • Both paths call upsert_movie_dates() appropriately

Investigation Status

✅ Enhanced Movie Detection: Working (Ick 2024 now processed)
✅ NFO Processing: Working (files get updated)
✅ File Timestamp Updates: Working (mtimes get set)
✅ Database Persistence: WORKING CORRECTLY - saves happen in both execution paths

📊 ENHANCED LOGGING SYSTEM (September 21, 2025) ✅

New Crystal-Clear Source Attribution

Problem: Logs didn't clearly show whether dates came from NFOGuard DB vs Radarr vs external APIs.

Solution: Added comprehensive logging with emojis and clear source identification.

New Logging Format

Priority Flow Display:

🔄 PRIORITY FLOW: NFOGuard DB → Radarr import → External APIs (digital,physical,theatrical) → Save to NFOGuard DB

NFOGuard Database Lookups:

🔍 NFOGuard DB lookup for tt1234567: FOUND - dateadded=2025-06-15T11:10:31-04:00, source=radarr:db.history.import
✅ MANUAL SCAN: Using existing NFOGuard database entry: 2025-06-15T11:10:31-04:00 (original source: radarr:db.history.import)

External Source Queries:

🔍 RADARR DATABASE: Checking import history for movie ID 123 (tt1234567)...
📦 RADARR DATABASE: Found import date=2025-07-20T14:30:00Z, source=radarr:db.history.import
🔍 EXTERNAL APIs: Trying digital release date fallback...
🌐 EXTERNAL APIs: Found digital release date=2025-06-15T00:00:00+00:00, source=tmdb:digital

Final Decision Tracking:

✅ FINAL CHOICE: Using Radarr import date 2025-07-20T14:30:00-04:00 from radarr:db.history.import
🎬 COMPLETED: Movie Name (2024) | Final date: 2025-07-20T14:30:00-04:00 | Final source: radarr:db.history.import

Source Attribution System

NFOGuard Database: nfoguard_db:radarr:db.history.import (shows original source)
Radarr Database: radarr:db.history.import (fresh query)
External APIs: tmdb:digital, omdb:dvd, etc.

Perfect Priority Flow Confirmed

  1. ✅ NFOGuard DB First: Check cached entries, use if available
  2. ✅ Radarr Import History: Query for real import dates if not cached
  3. ✅ External APIs: TMDB/OMDb fallback with configurable priority (RELEASE_DATE_PRIORITY)
  4. ✅ Save to NFOGuard DB: Cache result for future lookups

This ensures we don't repeatedly query Radarr/external APIs for the same movies!

🚀 MANUAL SCAN API COMMANDS

Available Scan Types

Movies Only:

curl -X POST "http://nfoguard:8080/manual/scan?scan_type=movies"

TV Shows Only:

curl -X POST "http://nfoguard:8080/manual/scan?scan_type=tv"

Both Movies AND TV Shows (Default):

curl -X POST "http://nfoguard:8080/manual/scan?scan_type=both"
# OR simply:
curl -X POST "http://nfoguard:8080/manual/scan"

Specific Path Scan:

curl -X POST "http://nfoguard:8080/manual/scan?path=/media/Movies/Movie.Name.2024&scan_type=movies"

What Each Scan Does

  • Movies: Processes all movie directories in MOVIE_PATHS, follows NFOGuard priority flow
  • TV: Processes all TV series in TV_PATHS, updates episode NFO files with proper dates
  • Both: Processes both movies and TV shows in sequence (recommended for full refresh)

All scans run in background and return immediately with status confirmation.

Movie IMDb ID Detection (✅ Enhanced)

Enhancement: Added comprehensive IMDb ID detection for movies without directory tags.

New Detection Methods:

  1. Directory Name: [imdb-tt123456] format (existing)
  2. Filename Patterns: IMDb ID in any video file name Movie.Name.2024.[tt123456].mkv
  3. NFO File Content: <uniqueid type="imdb">tt123456</uniqueid> (NEW)
  4. Legacy NFO Tags: <imdbid> and <imdb> tags (NEW)

Implementation:

  • Added find_movie_imdb_id() function that tries all methods in priority order
  • Extended parse_imdb_from_nfo() to handle XML parsing
  • Updated movie processing to use comprehensive detection

Example: Ick (2024) directory without IMDb in folder name will now be detected via NFO file content.

NFOGuard Development Summary

Current Status: v1.6.6 - Docker Plugin Deployment! 🐳

Latest Updates (v1.6.6) - September 19, 2025

  • 🐳 DOCKER PLUGIN DEPLOYMENT: Added automatic Emby plugin deployment via Docker
  • 📦 DLL PACKAGING: NFOGuard.Emby.Plugin.dll now packaged with Docker image
  • 🔗 BIND MOUNT SUPPORT: Primary method using bind mount to Emby plugins directory
  • ⚙️ ENVIRONMENT FALLBACK: Secondary method using EMBY_PLUGINS_PATH environment variable
  • 🚀 AUTO-UPDATE: Plugin automatically updates when new Docker image is deployed
  • 📋 CONFIGURATION: Added .env.example with plugin deployment settings
  • 🔄 STARTUP SCRIPT: Container automatically deploys plugin on startup
  • 🎯 WORKFLOW INTEGRATION: Compatible with Gitea → DockerHub deployment pipeline

Docker Plugin Deployment Flow

Bind Mount Method (Recommended):

  1. Set EMBY_PLUGINS_PATH=/path/to/emby/plugins in .env
  2. Docker bind mounts host directory to /emby-plugins in container
  3. On startup, NFOGuard copies DLL to mounted directory
  4. Plugin updates automatically with each new image deployment

Benefits:

  • Works with separate Emby/NFOGuard containers
  • Compatible with native Emby installations
  • Secure - only exposes plugins directory
  • Automatic updates via CI/CD pipeline

Previous Status: v1.6.0 - NFO-Based Identification! 📄

Latest Updates (v1.6.0) - September 16, 2025

  • 📄 NFO IDENTIFICATION: Added support for identifying movies/TV shows from NFO files
  • 🔄 DUAL METHOD: Directory name IMDb IDs (primary) + NFO file IMDb IDs (fallback)
  • 📋 NFO FORMATS: Supports <uniqueid type="imdb">, <imdbid>, and <imdb> XML tags
  • 🎬 MOVIE SUPPORT: Reads movie.nfo files for IMDb ID extraction
  • 📺 TV SUPPORT: Reads tvshow.nfo files for series identification
  • ✅ SMART FALLBACK: Only checks NFO files when directory name lacks IMDb ID
  • 📝 DOCUMENTATION: Updated README with NFO identification examples and formats
  • 🔄 NFO RENAMING: Added smart episode NFO file renaming to standardized S##E##.nfo format
  • 🎯 SONARR INTEGRATION: Detects existing NFO files created by Sonarr/other tools, extracts metadata, and renames to standard format
  • 📁 METADATA PRESERVATION: Preserves all existing metadata when renaming non-standard NFO files
  • 🧹 CLEANUP: Removes old non-standard NFO files after successful rename to prevent duplicates

Previous Updates (v1.5.5) - September 15, 2025

  • 🔧 DATABASE BUG FIX: Fixed upsert_movie_dates to be a proper upsert operation
  • 💾 DATA INTEGRITY: Manual scans now properly save dateadded to database
  • 🎯 ROOT CAUSE: Fixed UPDATE-only operation that was leaving dateadded fields NULL
  • ✅ WEBHOOK FIX: Webhooks now find existing database entries correctly
  • 📺 TV SHOWS: NOT affected - already using proper INSERT OR REPLACE operations

Previous Updates (v1.5.4) - September 15, 2025

  • 🎯 WEBHOOK DATE FIX: Fixed webhook processing to use proper date decision logic
  • 🔍 SMART FALLBACK: Webhooks now check database → Radarr import dates → release dates → timestamp
  • ❌ ISSUE RESOLVED: Upgrade webhooks no longer incorrectly use current timestamp when database exists
  • ✅ PROPER FLOW: Webhook mode now uses same date logic as manual scans for missing entries

Previous Updates (v1.5.3) - September 15, 2025

  • 🔧 RADARR RENAME FIX: Added missing event type filtering to Radarr webhooks
  • ✅ CONSISTENCY: Radarr now matches Sonarr's event handling (Download, Upgrade, Rename)
  • 🎯 FIXED ISSUE: Radarr rename events now properly trigger NFO updates with correct dates
  • 🔄 WORKFLOW: Both Sonarr and Radarr now follow identical webhook processing flows

Previous Updates (v1.5.2)

  • 🔇 TVDB WARNINGS: Added SUPPRESS_TVDB_WARNINGS configuration option
  • ✅ CLEAN LOGS: Can now suppress non-critical TVDB API failure warnings
  • 🧹 PRODUCTION: Cleaner logs for production monitoring
  • 🎯 FUNCTIONAL: System works perfectly without TVDB (uses IMDb/Sonarr data)

TVDB Warning Suppression

Added new environment variable:

# Suppress TVDB API warnings (true/false) - TVDB failures are common and non-critical
SUPPRESS_TVDB_WARNINGS=true

Why Suppress TVDB Warnings?

  • ❌ Common Failures: Many series don't exist in TVDB database
  • ✅ Non-Critical: NFOGuard works perfectly without TVDB data
  • 📊 Primary Data: Uses IMDb IDs and Sonarr metadata (more reliable)
  • 🧹 Log Noise: Reduces repetitive warning messages

Before vs After

Before (noisy):

INFO: All episodes found in cache
WARNING: GET https://api4.thetvdb.com/v4/search/remoteid?remoteId=tt0804484&type=series failed: HTTP Error 400: Bad Request
INFO: Completed processing TV series: Foundation (2021) [imdb-tt0804484]
WARNING: GET https://api4.thetvdb.com/v4/search/remoteid?remoteId=tt9737326&type=series failed: HTTP Error 400: Bad Request

After (clean):

INFO: All episodes found in cache
INFO: Completed processing TV series: Foundation (2021) [imdb-tt0804484]
INFO: Completed processing TV series: Invasion (2021) [imdb-tt9737326]

Your Configuration

DEBUG=false                    # Clean production logging
PATH_DEBUG=false              # No path mapping debug
SUPPRESS_TVDB_WARNINGS=true   # Hide non-critical TVDB failures

System Status

  • ✅ Processing: All TV series processing successfully
  • ✅ Episodes: All episodes found and processed
  • ✅ NFO Files: Being created/updated with proper metadata
  • ✅ Functionality: TVDB failures don't affect operation

🔇 NFOGuard v1.5.2 - Ultra-clean production logs! 🔇

Copy SUPPRESS_TVDB_WARNINGS=true to your server .env file and restart!

🔧 FIXED: Database Upsert Bug (September 15, 2025) - CRITICAL

Issue Discovery

The Real Root Cause: Manual scans were NOT actually saving dateadded to database due to faulty upsert operation!

Problem Details

Database Method Bug in core/database.py:173-182:

  • upsert_movie_dates() was doing UPDATE-only, not proper upsert
  • If movie row didn't exist properly, UPDATE would affect 0 rows
  • dateadded field remained NULL even after manual scan
  • Webhooks found database entries but with NULL dateadded → fell back to current timestamp

🔧 Fix Applied

  • Location: core/database.py:173-186
  • Changed: UPDATE-only → proper INSERT OR REPLACE upsert
  • Result: Manual scans now properly save dateadded to database
  • Webhook Impact: Now finds existing dates correctly, no more false timestamps

Before vs After Database Operations

Before (BROKEN):

UPDATE movies SET dateadded = '2025-06-15...' WHERE imdb_id = 'tt123';
-- If row doesn't exist properly: 0 rows affected, dateadded stays NULL

After (FIXED):

INSERT OR REPLACE INTO movies (...) VALUES (...);
-- Always works: Creates row OR replaces with all data including dateadded

🎯 FIXED: Webhook Date Logic Issue (September 15, 2025)

Issue Discovery

Upgrade webhooks were incorrectly using current timestamp (webhook:first_seen) instead of checking Radarr for proper import dates when no database entry existed.

Example Problem:

  • Database populated via manual scan: 2025-06-15T11:10:31-04:00 (radarr:db.history.import)
  • Upgrade webhook comes in: 2025-09-15T12:04:23-04:00 (webhook:first_seen) ❌ WRONG!

Root Cause

Webhook mode was bypassing the full date decision logic (_decide_movie_dates) and jumping straight to current timestamp as fallback.

✅ Fix Applied

  • Location: nfoguard.py:975-990
  • Logic: Webhook mode now uses same date priority as manual scans:
    1. Database first (if exists)
    2. Full date logic (Radarr import → release dates → fallbacks)
    3. Current timestamp (only as absolute last resort)

Result

Webhook upgrades now properly find and use existing Radarr import dates instead of creating false "first seen" timestamps.

🔧 FIXED: Radarr Rename Event Issue (September 15, 2025)

Issue Resolution

COMPLETED: Fixed missing Radarr rename event handling by adding proper event type filtering.

✅ Both Sonarr & Radarr Webhook Handling - NOW WORKING CORRECTLY

  • Sonarr Location: nfoguard.py:1429
  • Radarr Location: nfoguard.py:1479 (FIXED)
  • Event Filtering: if event_type not in ["Download", "Upgrade", "Rename"]:
  • ✅ STATUS: Rename events ARE processed for both systems
  • ✅ BEHAVIOR: Both systems now update NFO files with proper date metadata on rename

What Was Fixed

  1. Added Event Type Filtering: Radarr webhooks now explicitly filter for supported events
  2. Consistent Processing: Both Sonarr and Radarr now follow identical webhook workflows
  3. Proper Rename Handling: Rename events trigger full NFO processing pipeline

📋 NFOGuard Webhook Workflow Documentation

🎬 Radarr Webhook Workflow

  1. Import Event:

    • Check database - if not seen before, current timestamp = truth of first import
    • Update movie.nfo with proper date metadata

  2. Rename Event:

    • Check database - if we have the date, use existing date
    • Update movie.nfo file as always (preserving original import date)

  3. Rename Event (No Database Entry):

    • Check Radarr database for earliest import date
    • If found, use earliest import date → update database + movie.nfo
    • If no import date found, use digital/theatrical release date (fallback logic)

📺 Sonarr Webhook Workflow

  1. Import Event:

    • Check database - if not seen before, current timestamp = truth of first import
    • Update episode.nfo with proper date metadata

  2. Rename Event:

    • Check database - if we have the date, use existing date
    • Update episode.nfo file as always (preserving original import date)

  3. Rename Event (No Database Entry):

    • Check Sonarr API for earliest import date
    • If found, use earliest import date → update database + episode.nfo
    • If no import date found, use air date (fallback logic)

🔄 Key Processing Logic

  • Database First: Always check NFOGuard database for existing dates
  • Import Truth: Webhook import events establish "source of truth" timestamps
  • Date Preservation: Rename events preserve original import dates, never overwrite
  • Smart Fallbacks: API queries → Release dates → Air dates → File dates (configurable)
  • Batch Processing: All webhooks go through batching system with configurable delays
Reference in New Issue View Git Blame Copy Permalink