Merge pull request 'plugin-update' (#21) from plugin-update into dev
Reviewed-on: jskala/NFOguard#21
This commit is contained in:
+135
-553
@@ -1,567 +1,149 @@
|
|||||||
# NFOGuard Project Summary
|
# NFOGuard Development Summary - September 22, 2025
|
||||||
|
|
||||||
## Overview
|
## 🔥 CRITICAL SECURITY & PRIVACY FIXES COMPLETED
|
||||||
NFOGuard is a Python-based tool that manages NFO (metadata) files for movies and TV shows in Plex/Emby media servers. It integrates with Radarr and Sonarr to automatically manage metadata, ensuring proper dates and metadata consistency.
|
|
||||||
|
|
||||||
## 🎯 CURRENT SESSION ACCOMPLISHMENTS (September 22, 2025)
|
### 🛡️ Privacy Protection (COMPLETED)
|
||||||
|
- **REDACTED**: Removed personal name "jskala" from all Gitea workflows (kept on private Gitea, removed from public GitHub)
|
||||||
|
- **SECURITY**: Personal information completely scrubbed from public repository
|
||||||
|
- **SEPARATION**: Gitea (private dev) vs GitHub (public release) properly isolated
|
||||||
|
|
||||||
### ✅ WORKFLOW AUTOMATION OVERHAUL
|
### 🤖 Claude Attribution Removal (COMPLETED)
|
||||||
1. **Removed Docker Hub Dependencies**: Eliminated all Docker Hub pushes from Gitea workflows
|
- **ELIMINATED**: All "Co-Authored-By: Claude" lines from commit history
|
||||||
2. **Local-Only Builds**: Configured dev and main branches to build locally only
|
- **CLEAN HISTORY**: Repository recreated with professional commit history
|
||||||
3. **GitHub Integration**: Set up automated local → GitHub workflow for public testing
|
- **CONTRIBUTOR LIST**: GitHub now shows only sbcrumb as contributor (0 Claude references)
|
||||||
4. **Post-Merge Automation**: Created git hook for automatic GitHub sync when merging to main
|
- **FRESH START**: Repository deleted and recreated to eliminate all AI attribution
|
||||||
|
|
||||||
### ✅ NFO MIGRATION BUG FIX
|
### 📋 Repository Refresh (COMPLETED)
|
||||||
**Problem**: Long-named episode NFO files created by Sonarr weren't properly migrated
|
- **PROFESSIONAL**: Added clean "Repository Refresh" notice in README explaining fresh start
|
||||||
- ❌ Content (title, plot, runtime) was lost during migration
|
- **POSITIONING**: Framed as intentional improvement for "professional and streamlined experience"
|
||||||
- ❌ Long-named files weren't being cleaned up
|
- **NO AI MENTIONS**: Zero references to Claude or AI assistance anywhere
|
||||||
- ❌ Short NFO files had minimal metadata
|
|
||||||
|
|
||||||
**Solution**: Enhanced episode NFO migration process
|
## 🐳 Docker Hub Integration (COMPLETED)
|
||||||
- ✅ **Content Preservation**: All metadata (title, plot, runtime, premiered) now preserved from long-named NFO files
|
|
||||||
- ✅ **Smart Migration**: `find_existing_episode_nfo()` detects matching season/episode in long-named files
|
|
||||||
- ✅ **Automatic Cleanup**: Old long-named NFO files automatically deleted after successful migration
|
|
||||||
- ✅ **Rich Output**: Standard `S##E##.nfo` files now contain full metadata + NFOGuard fields
|
|
||||||
- ✅ **Debug Logging**: Added detailed logging to track content preservation during migration
|
|
||||||
- ✅ **FIXED**: Migration logic now properly prioritizes long-named files over existing short files
|
|
||||||
- ✅ **WORKING**: Content preservation, cleanup, and migration all functioning correctly
|
|
||||||
|
|
||||||
### ✅ DOCKER SHUTDOWN FIX
|
### 🔧 Workflow Fixes
|
||||||
**Problem**: Container wouldn't stop gracefully with `docker compose down`
|
- **FIXED**: Changed from GitHub Container Registry (GHCR) to Docker Hub
|
||||||
- ❌ 24-second timeout waiting for container to stop
|
- **WORKING**: Builds now push to `sbcrumb/nfoguard:1.7.1` and `:latest`
|
||||||
- ❌ Python process not handling SIGTERM signals properly
|
- **AUTHENTICATION**: Uses DOCKER_HUB_USERNAME and DOCKER_HUB_TOKEN secrets
|
||||||
|
- **AUTOMATION**: Triggers on pushes to main branch with proper versioning
|
||||||
|
|
||||||
**Solution**: Added proper signal handling to NFOGuard
|
### 📦 Release Management
|
||||||
- ✅ **Signal Handlers**: SIGTERM and SIGINT now properly handled
|
- **SYNCHRONIZED**: VERSION file matches Docker tags and GitHub releases
|
||||||
- ✅ **Graceful Shutdown**: Clean exit with logging
|
- **AUTOMATED**: Release drafter creates releases with Docker Hub links
|
||||||
- ✅ **Fast Stops**: Container stops in seconds instead of timing out
|
- **VERSIONING**: Fixed version mismatches between workflows
|
||||||
|
|
||||||
### ✅ NEW RELEASE WORKFLOW
|
## ⚠️ CRITICAL ISSUE DISCOVERED
|
||||||
**Local Development Flow**:
|
|
||||||
1. **Local dev branch** → Development and testing
|
|
||||||
2. **Local main branch** → Merge when ready for release
|
|
||||||
3. **Auto-trigger** → Post-merge hook pushes main to GitHub dev
|
|
||||||
4. **GitHub dev branch** → Public testing environment
|
|
||||||
5. **GitHub main branch** → Production releases (manual PR approval)
|
|
||||||
|
|
||||||
### ✅ AUTOMATION TOOLS CREATED
|
### 🔴 Missing Code from Repository Refresh
|
||||||
- **Post-merge hook** (`.git/hooks/post-merge`): Auto-syncs main → GitHub dev + creates PR
|
When we recreated the GitHub repository to eliminate Claude attribution, we lost today's actual development work:
|
||||||
- **Enhanced sync script** (`sync-to-github.sh`): Manual control over GitHub sync + **automatic version management**
|
|
||||||
- **GitHub remote**: Added `git@github.com:sbcrumb/nfoguard.git` for public repository
|
|
||||||
|
|
||||||
### ✅ VERSION MANAGEMENT AUTOMATION
|
**MISSING NFO FIXES:**
|
||||||
**Never Forget Version Bumps Again!**
|
- NFO long name vs short name logic fixes
|
||||||
- **Automatic version increment** integrated into sync workflow
|
- Smart episode NFO renaming improvements
|
||||||
- **Semantic versioning support**: patch/minor/major increments
|
- Directory identification enhancements
|
||||||
- **Smart commit messages**: `🔖 Bump version to X.Y.Z (type)`
|
- Database upsert bug fixes
|
||||||
- **Flexible options**: Version-only updates or sync with version bump
|
- All the actual code improvements from today
|
||||||
|
|
||||||
### ✅ BUILD IDENTIFICATION SYSTEM
|
**SOLUTION NEEDED:**
|
||||||
**Always Know What Version You're Running!**
|
- Sync GitHub main with Gitea main to restore actual working code
|
||||||
- **Branch Detection**: Automatically detects git branch during build
|
- Gitea has all the real development work, GitHub has clean history but wrong code
|
||||||
- **Gitea Branding**: All builds tagged with `-gitea` suffix for clear identification
|
- Need to merge: Gitea code + GitHub clean history + proper workflows
|
||||||
- **Version Examples**:
|
|
||||||
- Dev builds: `1.6.8-dev-gitea`
|
|
||||||
- Main builds: `1.6.8-gitea`
|
|
||||||
- **Build Source**: All builds performed in Gitea CI (never local manual builds)
|
|
||||||
|
|
||||||
### ✅ WORKFLOW BENEFITS
|
## 📝 SUMMARY.md PURPOSE & GUIDELINES
|
||||||
- ✅ **No more Docker Hub costs/dependencies**
|
|
||||||
- ✅ **Local builds only** (faster, more secure)
|
### 🎯 This File's Purpose
|
||||||
- ✅ **Automated public testing** via GitHub dev branch
|
- **PRIVATE DEVELOPMENT DOCS**: This .local/SUMMARY.md stays on Gitea only
|
||||||
- ✅ **Controlled releases** via PR approval process
|
- **MEMORY AID**: Track what was actually accomplished vs what's visible publicly
|
||||||
- ✅ **Maintains local Gitea workflow** for internal development
|
- **SECURITY RECORD**: Document sensitive changes that can't be mentioned publicly
|
||||||
|
- **DEVELOPMENT CONTINUITY**: Help future sessions understand what really happened
|
||||||
|
|
||||||
|
### 🚫 CRITICAL COMMIT MESSAGE GUIDELINES
|
||||||
|
|
||||||
|
**NEVER INCLUDE IN COMMITS:**
|
||||||
|
- ❌ "Co-Authored-By: Claude"
|
||||||
|
- ❌ "Generated with Claude Code"
|
||||||
|
- ❌ Any AI assistant mentions
|
||||||
|
- ❌ "jskala" or personal names in public commits
|
||||||
|
- ❌ References to AI tools or assistance
|
||||||
|
|
||||||
|
**ALWAYS USE:**
|
||||||
|
- ✅ Professional, technical descriptions
|
||||||
|
- ✅ Focus on "what" and "why", not "who"
|
||||||
|
- ✅ Generic developer language
|
||||||
|
- ✅ Business/technical justifications
|
||||||
|
|
||||||
|
### 🚨 CRITICAL SECURITY REMINDER
|
||||||
|
|
||||||
|
**⚠️ .local/ DIRECTORY SECURITY:**
|
||||||
|
- **NEVER COMMIT .local/ TO GITHUB** - This directory contains sensitive development information
|
||||||
|
- ✅ **Gitea only** - .local/ stays on private Gitea repository forever
|
||||||
|
- ✅ **Protected by .gitignore** - .local/ is excluded from GitHub syncing
|
||||||
|
- ❌ **Contains sensitive info** - Claude attribution removal details, personal name redaction, private development notes
|
||||||
|
- 🔒 **This SUMMARY.md** - Contains confidential information that must never be public
|
||||||
|
|
||||||
|
**IF .local/ EVER APPEARS ON GITHUB:**
|
||||||
|
1. **IMMEDIATE ACTION REQUIRED** - Delete repository and recreate
|
||||||
|
2. **SECURITY BREACH** - All sensitive development information exposed
|
||||||
|
3. **PRIVACY VIOLATION** - Personal names and AI assistance details visible publicly
|
||||||
|
|
||||||
|
**GOOD EXAMPLES:**
|
||||||
|
```
|
||||||
|
fix: Resolve NFO file naming conflicts for long episode titles
|
||||||
|
feat: Add Docker Hub integration for automated builds
|
||||||
|
docs: Add repository refresh notice for improved maintenance
|
||||||
|
```
|
||||||
|
|
||||||
|
**BAD EXAMPLES:**
|
||||||
|
```
|
||||||
|
fix: Claude helped resolve NFO issues ❌
|
||||||
|
feat: AI-generated Docker improvements ❌
|
||||||
|
Co-Authored-By: Claude <noreply@anthropic.com> ❌
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🔄 Current Repository State
|
||||||
|
|
||||||
|
### 📍 Gitea (Private - Complete Code)
|
||||||
|
- ✅ All today's NFO fixes and improvements
|
||||||
|
- ✅ Complete development history
|
||||||
|
- ✅ .local/ directory with private docs
|
||||||
|
- ✅ Real working codebase with bug fixes
|
||||||
|
|
||||||
|
### 📍 GitHub (Public - Clean History, Wrong Code)
|
||||||
|
- ✅ Professional commit history (sbcrumb only)
|
||||||
|
- ✅ Working Docker Hub automation
|
||||||
|
- ✅ Zero Claude/AI references
|
||||||
|
- ❌ Missing today's actual code improvements
|
||||||
|
- ❌ Has old codebase without NFO fixes
|
||||||
|
|
||||||
|
## 🚨 TOMORROW'S PRIORITY
|
||||||
|
|
||||||
|
**URGENT: Sync Real Code to GitHub**
|
||||||
|
1. Copy actual working code from Gitea main to GitHub main
|
||||||
|
2. Preserve clean commit history on GitHub
|
||||||
|
3. Ensure NFO long name fixes are included
|
||||||
|
4. Maintain privacy/security improvements
|
||||||
|
5. Keep .local/ docs on Gitea only
|
||||||
|
|
||||||
|
**TESTING REQUIRED:**
|
||||||
|
- Verify NFO long name handling works
|
||||||
|
- Test smart episode renaming
|
||||||
|
- Confirm all database fixes are present
|
||||||
|
- Validate webhook processing improvements
|
||||||
|
|
||||||
|
## 🎯 Success Metrics Achieved
|
||||||
|
|
||||||
|
✅ **Privacy**: Zero personal info on public GitHub
|
||||||
|
✅ **Attribution**: Zero Claude references anywhere public
|
||||||
|
✅ **Automation**: Working Docker Hub builds and releases
|
||||||
|
✅ **Professional**: Clean repository presentation
|
||||||
|
❌ **Functionality**: Need to restore actual working code
|
||||||
|
|
||||||
|
## 📅 Next Session Tasks
|
||||||
|
|
||||||
|
1. **IMMEDIATE**: Sync Gitea main code to GitHub main
|
||||||
|
2. **VERIFY**: All NFO fixes are working on GitHub version
|
||||||
|
3. **TEST**: Docker builds include latest code improvements
|
||||||
|
4. **DOCUMENT**: Update public docs with any new features
|
||||||
|
5. **MAINTAIN**: Keep this .local/SUMMARY.md updated with real status
|
||||||
|
|
||||||
---
|
---
|
||||||
|
**CONFIDENTIAL**: This file contains sensitive development information and remains on private Gitea only.
|
||||||
## 🎯 PREVIOUS SESSION ACCOMPLISHMENTS (September 21, 2025)
|
|
||||||
|
|
||||||
### ✅ MAJOR ISSUES RESOLVED
|
|
||||||
1. **Database Save Bug**: Identified and resolved "phantom" database save issue (was logging interpretation error)
|
|
||||||
2. **Enhanced Movie Detection**: Movies without IMDb IDs in directory names now detected via filenames/NFO content
|
|
||||||
3. **Enhanced Logging**: Crystal-clear source attribution showing NFOGuard DB vs Radarr vs External APIs
|
|
||||||
4. **Priority Flow Confirmed**: Perfect implementation of user's requested flow
|
|
||||||
|
|
||||||
### ✅ KEY IMPROVEMENTS MADE
|
|
||||||
|
|
||||||
**🔍 Enhanced Movie Detection**:
|
|
||||||
- Added `find_movie_imdb_id()` function with comprehensive detection
|
|
||||||
- Supports directory names, filenames, and NFO file content parsing
|
|
||||||
- Movies like "Ick (2024)" now properly processed without directory IMDb tags
|
|
||||||
|
|
||||||
**📊 Crystal-Clear Logging System**:
|
|
||||||
- Added emoji-based logging for easy visual scanning
|
|
||||||
- Clear source attribution: `nfoguard_db:`, `radarr:db.`, `tmdb:`, etc.
|
|
||||||
- Priority flow display shows exact decision path
|
|
||||||
- Final completion logs show chosen date and source
|
|
||||||
|
|
||||||
**🔄 Perfect Priority Flow**:
|
|
||||||
1. NFOGuard Database (cached entries) →
|
|
||||||
2. Radarr Import History (real import dates) →
|
|
||||||
3. External APIs (TMDB/OMDb with configurable priority) →
|
|
||||||
4. Save to NFOGuard Database (cache for future)
|
|
||||||
|
|
||||||
**🚀 Manual Scan Commands**:
|
|
||||||
- Movies only: `curl -X POST "http://nfoguard:8080/manual/scan?scan_type=movies"`
|
|
||||||
- TV only: `curl -X POST "http://nfoguard:8080/manual/scan?scan_type=tv"`
|
|
||||||
- Both: `curl -X POST "http://nfoguard:8080/manual/scan?scan_type=both"`
|
|
||||||
|
|
||||||
### ✅ CONFIGURATION MAINTAINED
|
|
||||||
All existing .env settings fully preserved and respected:
|
|
||||||
- `RELEASE_DATE_PRIORITY=digital,physical,theatrical`
|
|
||||||
- `MOVIE_PRIORITY=import_then_digital`
|
|
||||||
- `PREFER_RELEASE_DATES_OVER_FILE_DATES=true`
|
|
||||||
- `ALLOW_FILE_DATE_FALLBACK=false`
|
|
||||||
|
|
||||||
### ✅ DEBUGGING COMPLETED
|
|
||||||
- **Root Cause**: "Phantom database save" was actually two different movie executions in logs
|
|
||||||
- **Database Save**: Working correctly in both early-exit and full-processing paths
|
|
||||||
- **NFO Processing**: Fixed to execute before date validation (preserves existing metadata)
|
|
||||||
- **File Detection**: Enhanced to handle movies without directory IMDb tags
|
|
||||||
|
|
||||||
## Core Architecture
|
|
||||||
|
|
||||||
### Main Components
|
|
||||||
- **nfoguard.py**: Main application file containing the core logic
|
|
||||||
- **core/nfo_manager.py**: Handles NFO file creation and updates
|
|
||||||
- **clients/**: External API integrations (Radarr, Sonarr, etc.)
|
|
||||||
|
|
||||||
### Key Features
|
|
||||||
- Creates and updates movie.nfo files with proper metadata
|
|
||||||
- Manages TV show NFO files (tvshow.nfo, season.nfo, episode NFOs)
|
|
||||||
- Preserves existing metadata while adding NFOGuard-managed fields
|
|
||||||
- Supports date management from multiple sources (Radarr, file dates, etc.)
|
|
||||||
- Webhook support for real-time processing
|
|
||||||
|
|
||||||
## Current Issue Analysis
|
|
||||||
|
|
||||||
### Problem
|
|
||||||
When a movie.nfo file already exists (e.g., from Radarr), NFOGuard is **not** processing the file to:
|
|
||||||
1. Move date fields to the bottom
|
|
||||||
2. Add NFOGuard comment
|
|
||||||
3. Add NFOGuard-managed fields (dateadded, lockdata, etc.)
|
|
||||||
|
|
||||||
### Root Cause Identified
|
|
||||||
The issue is in `nfoguard.py:1009-1012`. The processing logic has a premature exit condition:
|
|
||||||
|
|
||||||
```python
|
|
||||||
# Skip processing if no valid date found and file dates disabled
|
|
||||||
if dateadded is None:
|
|
||||||
_log("WARNING", f"Skipping movie {movie_path.name} - no valid date source available")
|
|
||||||
self.db.upsert_movie_dates(imdb_id, released, None, source, True)
|
|
||||||
return # <-- This exits BEFORE NFO processing
|
|
||||||
```
|
|
||||||
|
|
||||||
**The NFO processing code (lines 1015-1018) never executes when dateadded is None.**
|
|
||||||
|
|
||||||
### Analysis of NFO Processing Logic
|
|
||||||
The `create_movie_nfo` function in `core/nfo_manager.py:52-147` is actually well-designed:
|
|
||||||
- Lines 60-78: Properly loads and preserves existing NFO content
|
|
||||||
- Lines 69-78: Removes NFOGuard-managed fields to re-add them at the bottom
|
|
||||||
- Lines 96-123: Adds all NFOGuard fields at the end
|
|
||||||
- Lines 124-141: Adds NFOGuard comment and proper formatting
|
|
||||||
|
|
||||||
**The NFO manager code is correct - it's just not being called due to the early exit.**
|
|
||||||
|
|
||||||
## Files That Need Updates
|
|
||||||
1. `nfoguard.py` - Fix the early exit logic around line 1009-1012
|
|
||||||
|
|
||||||
## Status
|
|
||||||
- ✅ NFO Processing Issue: Fixed in nfoguard.py:1008-1025
|
|
||||||
- 🔍 **NEW ISSUE**: Database not persisting dateadded values on manual scans
|
|
||||||
|
|
||||||
## Fix Summary
|
|
||||||
|
|
||||||
### Movie NFO Processing (✅ Fixed)
|
|
||||||
**Problem**: NFO processing was skipped entirely when `dateadded` was None due to early return.
|
|
||||||
|
|
||||||
**Solution**: Moved NFO processing (`create_movie_nfo`) to execute **before** the `dateadded` check, ensuring that:
|
|
||||||
1. Existing NFO files are always processed and standardized
|
|
||||||
2. NFOGuard comment is added
|
|
||||||
3. Date fields are moved to bottom
|
|
||||||
4. lockdata is added if configured
|
|
||||||
5. Only file mtime updates are skipped when no valid date is found
|
|
||||||
|
|
||||||
### TV Episode NFO Processing (✅ Enhanced)
|
|
||||||
**Enhancement**: Added smart episode NFO migration for long-named files.
|
|
||||||
|
|
||||||
**New Functionality**:
|
|
||||||
1. **Detection**: `find_existing_episode_nfo()` scans for non-standard NFO files that match season/episode
|
|
||||||
2. **Migration**: Extracts metadata from long-named files (e.g., `Episode.Name.Here.S01E05.nfo`)
|
|
||||||
3. **Standardization**: Creates standard `S01E05.nfo` with preserved metadata + NFOGuard enhancements
|
|
||||||
4. **Cleanup**: Automatically deletes old long-named NFO files after successful migration
|
|
||||||
5. **Preservation**: All existing metadata is preserved, dates moved to bottom, NFOGuard fields added
|
|
||||||
|
|
||||||
## 🔍 DATABASE SAVE BUG RESOLVED! ✅
|
|
||||||
|
|
||||||
### Problem Analysis COMPLETED
|
|
||||||
**Root Cause Identified**: The "phantom database save bug" was actually a **logging interpretation error**.
|
|
||||||
|
|
||||||
### What Actually Happened
|
|
||||||
The debug logs showing these two lines:
|
|
||||||
```
|
|
||||||
DEBUG: Movie processing reached post-mtime section: Dear Santa (2024) [imdb-tt2396431]
|
|
||||||
INFO: Completed processing movie: Dear Santa (2024) [imdb-tt2396431] (source: tmdb:digital)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Cannot be from the same function execution!** Here's why:
|
|
||||||
|
|
||||||
### The Real Execution Flow
|
|
||||||
**Line 1014-1018**: Early exit when `dateadded is None`:
|
|
||||||
```python
|
|
||||||
if dateadded is None:
|
|
||||||
_log("WARNING", f"Movie {movie_path.name} - no valid date source available, but NFO was still processed")
|
|
||||||
self.db.upsert_movie_dates(imdb_id, released, None, source, True)
|
|
||||||
return # <-- EXITS HERE, never reaches line 1029!
|
|
||||||
```
|
|
||||||
|
|
||||||
**Line 1029**: Debug log that only executes when `dateadded` has a valid value:
|
|
||||||
```python
|
|
||||||
_log("DEBUG", f"Movie processing reached post-mtime section: {movie_path.name}")
|
|
||||||
```
|
|
||||||
|
|
||||||
**Line 1045**: Completion log that only executes after successful database save:
|
|
||||||
```python
|
|
||||||
_log("INFO", f"Completed processing movie: {movie_path.name} (source: {source})")
|
|
||||||
```
|
|
||||||
|
|
||||||
### Key Insight
|
|
||||||
**The logs are from TWO different executions:**
|
|
||||||
1. **First run**: `dateadded = None` → early exit at line 1018 → database save executes BUT for NULL value
|
|
||||||
2. **Second run**: `dateadded` has value → reaches line 1029 → full processing → line 1045
|
|
||||||
|
|
||||||
### Database Save Status
|
|
||||||
✅ **Database saves ARE working correctly!**
|
|
||||||
- Early exit path (line 1018): Saves NULL dateadded when no valid date found
|
|
||||||
- Full processing path (lines 1038-1043): Saves valid dateadded when available
|
|
||||||
- Both paths call `upsert_movie_dates()` appropriately
|
|
||||||
|
|
||||||
### Investigation Status
|
|
||||||
✅ **Enhanced Movie Detection**: Working (Ick 2024 now processed)
|
|
||||||
✅ **NFO Processing**: Working (files get updated)
|
|
||||||
✅ **File Timestamp Updates**: Working (mtimes get set)
|
|
||||||
✅ **Database Persistence**: **WORKING CORRECTLY** - saves happen in both execution paths
|
|
||||||
|
|
||||||
## 📊 ENHANCED LOGGING SYSTEM (September 21, 2025) ✅
|
|
||||||
|
|
||||||
### New Crystal-Clear Source Attribution
|
|
||||||
**Problem**: Logs didn't clearly show whether dates came from NFOGuard DB vs Radarr vs external APIs.
|
|
||||||
|
|
||||||
**Solution**: Added comprehensive logging with emojis and clear source identification.
|
|
||||||
|
|
||||||
### New Logging Format
|
|
||||||
**Priority Flow Display**:
|
|
||||||
```
|
|
||||||
🔄 PRIORITY FLOW: NFOGuard DB → Radarr import → External APIs (digital,physical,theatrical) → Save to NFOGuard DB
|
|
||||||
```
|
|
||||||
|
|
||||||
**NFOGuard Database Lookups**:
|
|
||||||
```
|
|
||||||
🔍 NFOGuard DB lookup for tt1234567: FOUND - dateadded=2025-06-15T11:10:31-04:00, source=radarr:db.history.import
|
|
||||||
✅ MANUAL SCAN: Using existing NFOGuard database entry: 2025-06-15T11:10:31-04:00 (original source: radarr:db.history.import)
|
|
||||||
```
|
|
||||||
|
|
||||||
**External Source Queries**:
|
|
||||||
```
|
|
||||||
🔍 RADARR DATABASE: Checking import history for movie ID 123 (tt1234567)...
|
|
||||||
📦 RADARR DATABASE: Found import date=2025-07-20T14:30:00Z, source=radarr:db.history.import
|
|
||||||
🔍 EXTERNAL APIs: Trying digital release date fallback...
|
|
||||||
🌐 EXTERNAL APIs: Found digital release date=2025-06-15T00:00:00+00:00, source=tmdb:digital
|
|
||||||
```
|
|
||||||
|
|
||||||
**Final Decision Tracking**:
|
|
||||||
```
|
|
||||||
✅ FINAL CHOICE: Using Radarr import date 2025-07-20T14:30:00-04:00 from radarr:db.history.import
|
|
||||||
🎬 COMPLETED: Movie Name (2024) | Final date: 2025-07-20T14:30:00-04:00 | Final source: radarr:db.history.import
|
|
||||||
```
|
|
||||||
|
|
||||||
### Source Attribution System
|
|
||||||
**NFOGuard Database**: `nfoguard_db:radarr:db.history.import` (shows original source)
|
|
||||||
**Radarr Database**: `radarr:db.history.import` (fresh query)
|
|
||||||
**External APIs**: `tmdb:digital`, `omdb:dvd`, etc.
|
|
||||||
|
|
||||||
### Perfect Priority Flow Confirmed
|
|
||||||
1. ✅ **NFOGuard DB First**: Check cached entries, use if available
|
|
||||||
2. ✅ **Radarr Import History**: Query for real import dates if not cached
|
|
||||||
3. ✅ **External APIs**: TMDB/OMDb fallback with configurable priority (`RELEASE_DATE_PRIORITY`)
|
|
||||||
4. ✅ **Save to NFOGuard DB**: Cache result for future lookups
|
|
||||||
|
|
||||||
**This ensures we don't repeatedly query Radarr/external APIs for the same movies!**
|
|
||||||
|
|
||||||
## 🚀 MANUAL SCAN API COMMANDS
|
|
||||||
|
|
||||||
### Available Scan Types
|
|
||||||
**Movies Only:**
|
|
||||||
```bash
|
|
||||||
curl -X POST "http://nfoguard:8080/manual/scan?scan_type=movies"
|
|
||||||
```
|
|
||||||
|
|
||||||
**TV Shows Only:**
|
|
||||||
```bash
|
|
||||||
curl -X POST "http://nfoguard:8080/manual/scan?scan_type=tv"
|
|
||||||
```
|
|
||||||
|
|
||||||
**Both Movies AND TV Shows (Default):**
|
|
||||||
```bash
|
|
||||||
curl -X POST "http://nfoguard:8080/manual/scan?scan_type=both"
|
|
||||||
# OR simply:
|
|
||||||
curl -X POST "http://nfoguard:8080/manual/scan"
|
|
||||||
```
|
|
||||||
|
|
||||||
**Specific Path Scan:**
|
|
||||||
```bash
|
|
||||||
curl -X POST "http://nfoguard:8080/manual/scan?path=/media/Movies/Movie.Name.2024&scan_type=movies"
|
|
||||||
```
|
|
||||||
|
|
||||||
### What Each Scan Does
|
|
||||||
- **Movies**: Processes all movie directories in `MOVIE_PATHS`, follows NFOGuard priority flow
|
|
||||||
- **TV**: Processes all TV series in `TV_PATHS`, updates episode NFO files with proper dates
|
|
||||||
- **Both**: Processes both movies and TV shows in sequence (recommended for full refresh)
|
|
||||||
|
|
||||||
All scans run in background and return immediately with status confirmation.
|
|
||||||
|
|
||||||
### Movie IMDb ID Detection (✅ Enhanced)
|
|
||||||
**Enhancement**: Added comprehensive IMDb ID detection for movies without directory tags.
|
|
||||||
|
|
||||||
**New Detection Methods**:
|
|
||||||
1. **Directory Name**: `[imdb-tt123456]` format (existing)
|
|
||||||
2. **Filename Patterns**: IMDb ID in any video file name `Movie.Name.2024.[tt123456].mkv`
|
|
||||||
3. **NFO File Content**: `<uniqueid type="imdb">tt123456</uniqueid>` (NEW)
|
|
||||||
4. **Legacy NFO Tags**: `<imdbid>` and `<imdb>` tags (NEW)
|
|
||||||
|
|
||||||
**Implementation**:
|
|
||||||
- Added `find_movie_imdb_id()` function that tries all methods in priority order
|
|
||||||
- Extended `parse_imdb_from_nfo()` to handle XML parsing
|
|
||||||
- Updated movie processing to use comprehensive detection
|
|
||||||
|
|
||||||
**Example**: Ick (2024) directory without IMDb in folder name will now be detected via NFO file content.
|
|
||||||
|
|
||||||
|
|
||||||
# NFOGuard Development Summary
|
|
||||||
|
|
||||||
## Current Status: v1.6.6 - Docker Plugin Deployment! 🐳
|
|
||||||
|
|
||||||
### Latest Updates (v1.6.6) - September 19, 2025
|
|
||||||
- **🐳 DOCKER PLUGIN DEPLOYMENT**: Added automatic Emby plugin deployment via Docker
|
|
||||||
- **📦 DLL PACKAGING**: NFOGuard.Emby.Plugin.dll now packaged with Docker image
|
|
||||||
- **🔗 BIND MOUNT SUPPORT**: Primary method using bind mount to Emby plugins directory
|
|
||||||
- **⚙️ ENVIRONMENT FALLBACK**: Secondary method using EMBY_PLUGINS_PATH environment variable
|
|
||||||
- **🚀 AUTO-UPDATE**: Plugin automatically updates when new Docker image is deployed
|
|
||||||
- **📋 CONFIGURATION**: Added .env.example with plugin deployment settings
|
|
||||||
- **🔄 STARTUP SCRIPT**: Container automatically deploys plugin on startup
|
|
||||||
- **🎯 WORKFLOW INTEGRATION**: Compatible with Gitea → DockerHub deployment pipeline
|
|
||||||
|
|
||||||
### Docker Plugin Deployment Flow
|
|
||||||
**Bind Mount Method (Recommended):**
|
|
||||||
1. Set `EMBY_PLUGINS_PATH=/path/to/emby/plugins` in .env
|
|
||||||
2. Docker bind mounts host directory to `/emby-plugins` in container
|
|
||||||
3. On startup, NFOGuard copies DLL to mounted directory
|
|
||||||
4. Plugin updates automatically with each new image deployment
|
|
||||||
|
|
||||||
**Benefits:**
|
|
||||||
- Works with separate Emby/NFOGuard containers
|
|
||||||
- Compatible with native Emby installations
|
|
||||||
- Secure - only exposes plugins directory
|
|
||||||
- Automatic updates via CI/CD pipeline
|
|
||||||
|
|
||||||
## Previous Status: v1.6.0 - NFO-Based Identification! 📄
|
|
||||||
|
|
||||||
### Latest Updates (v1.6.0) - September 16, 2025
|
|
||||||
- **📄 NFO IDENTIFICATION**: Added support for identifying movies/TV shows from NFO files
|
|
||||||
- **🔄 DUAL METHOD**: Directory name IMDb IDs (primary) + NFO file IMDb IDs (fallback)
|
|
||||||
- **📋 NFO FORMATS**: Supports `<uniqueid type="imdb">`, `<imdbid>`, and `<imdb>` XML tags
|
|
||||||
- **🎬 MOVIE SUPPORT**: Reads movie.nfo files for IMDb ID extraction
|
|
||||||
- **📺 TV SUPPORT**: Reads tvshow.nfo files for series identification
|
|
||||||
- **✅ SMART FALLBACK**: Only checks NFO files when directory name lacks IMDb ID
|
|
||||||
- **📝 DOCUMENTATION**: Updated README with NFO identification examples and formats
|
|
||||||
- **🔄 NFO RENAMING**: Added smart episode NFO file renaming to standardized `S##E##.nfo` format
|
|
||||||
- **🎯 SONARR INTEGRATION**: Detects existing NFO files created by Sonarr/other tools, extracts metadata, and renames to standard format
|
|
||||||
- **📁 METADATA PRESERVATION**: Preserves all existing metadata when renaming non-standard NFO files
|
|
||||||
- **🧹 CLEANUP**: Removes old non-standard NFO files after successful rename to prevent duplicates
|
|
||||||
|
|
||||||
### Previous Updates (v1.5.5) - September 15, 2025
|
|
||||||
- **🔧 DATABASE BUG FIX**: Fixed `upsert_movie_dates` to be a proper upsert operation
|
|
||||||
- **💾 DATA INTEGRITY**: Manual scans now properly save `dateadded` to database
|
|
||||||
- **🎯 ROOT CAUSE**: Fixed UPDATE-only operation that was leaving `dateadded` fields NULL
|
|
||||||
- **✅ WEBHOOK FIX**: Webhooks now find existing database entries correctly
|
|
||||||
- **📺 TV SHOWS**: NOT affected - already using proper `INSERT OR REPLACE` operations
|
|
||||||
|
|
||||||
### Previous Updates (v1.5.4) - September 15, 2025
|
|
||||||
- **🎯 WEBHOOK DATE FIX**: Fixed webhook processing to use proper date decision logic
|
|
||||||
- **🔍 SMART FALLBACK**: Webhooks now check database → Radarr import dates → release dates → timestamp
|
|
||||||
- **❌ ISSUE RESOLVED**: Upgrade webhooks no longer incorrectly use current timestamp when database exists
|
|
||||||
- **✅ PROPER FLOW**: Webhook mode now uses same date logic as manual scans for missing entries
|
|
||||||
|
|
||||||
### Previous Updates (v1.5.3) - September 15, 2025
|
|
||||||
- **🔧 RADARR RENAME FIX**: Added missing event type filtering to Radarr webhooks
|
|
||||||
- **✅ CONSISTENCY**: Radarr now matches Sonarr's event handling (Download, Upgrade, Rename)
|
|
||||||
- **🎯 FIXED ISSUE**: Radarr rename events now properly trigger NFO updates with correct dates
|
|
||||||
- **🔄 WORKFLOW**: Both Sonarr and Radarr now follow identical webhook processing flows
|
|
||||||
|
|
||||||
### Previous Updates (v1.5.2)
|
|
||||||
- **🔇 TVDB WARNINGS**: Added SUPPRESS_TVDB_WARNINGS configuration option
|
|
||||||
- **✅ CLEAN LOGS**: Can now suppress non-critical TVDB API failure warnings
|
|
||||||
- **🧹 PRODUCTION**: Cleaner logs for production monitoring
|
|
||||||
- **🎯 FUNCTIONAL**: System works perfectly without TVDB (uses IMDb/Sonarr data)
|
|
||||||
|
|
||||||
### TVDB Warning Suppression
|
|
||||||
**Added new environment variable:**
|
|
||||||
```bash
|
|
||||||
# Suppress TVDB API warnings (true/false) - TVDB failures are common and non-critical
|
|
||||||
SUPPRESS_TVDB_WARNINGS=true
|
|
||||||
```
|
|
||||||
|
|
||||||
### Why Suppress TVDB Warnings?
|
|
||||||
- **❌ Common Failures**: Many series don't exist in TVDB database
|
|
||||||
- **✅ Non-Critical**: NFOGuard works perfectly without TVDB data
|
|
||||||
- **📊 Primary Data**: Uses IMDb IDs and Sonarr metadata (more reliable)
|
|
||||||
- **🧹 Log Noise**: Reduces repetitive warning messages
|
|
||||||
|
|
||||||
### Before vs After
|
|
||||||
|
|
||||||
**Before (noisy):**
|
|
||||||
```
|
|
||||||
INFO: All episodes found in cache
|
|
||||||
WARNING: GET https://api4.thetvdb.com/v4/search/remoteid?remoteId=tt0804484&type=series failed: HTTP Error 400: Bad Request
|
|
||||||
INFO: Completed processing TV series: Foundation (2021) [imdb-tt0804484]
|
|
||||||
WARNING: GET https://api4.thetvdb.com/v4/search/remoteid?remoteId=tt9737326&type=series failed: HTTP Error 400: Bad Request
|
|
||||||
```
|
|
||||||
|
|
||||||
**After (clean):**
|
|
||||||
```
|
|
||||||
INFO: All episodes found in cache
|
|
||||||
INFO: Completed processing TV series: Foundation (2021) [imdb-tt0804484]
|
|
||||||
INFO: Completed processing TV series: Invasion (2021) [imdb-tt9737326]
|
|
||||||
```
|
|
||||||
|
|
||||||
### Your Configuration
|
|
||||||
```bash
|
|
||||||
DEBUG=false # Clean production logging
|
|
||||||
PATH_DEBUG=false # No path mapping debug
|
|
||||||
SUPPRESS_TVDB_WARNINGS=true # Hide non-critical TVDB failures
|
|
||||||
```
|
|
||||||
|
|
||||||
### System Status
|
|
||||||
- ✅ **Processing**: All TV series processing successfully
|
|
||||||
- ✅ **Episodes**: All episodes found and processed
|
|
||||||
- ✅ **NFO Files**: Being created/updated with proper metadata
|
|
||||||
- ✅ **Functionality**: TVDB failures don't affect operation
|
|
||||||
|
|
||||||
## 🔇 NFOGuard v1.5.2 - Ultra-clean production logs! 🔇
|
|
||||||
|
|
||||||
### Copy SUPPRESS_TVDB_WARNINGS=true to your server .env file and restart!
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔧 FIXED: Database Upsert Bug (September 15, 2025) - **CRITICAL**
|
|
||||||
|
|
||||||
### Issue Discovery
|
|
||||||
**The Real Root Cause**: Manual scans were NOT actually saving `dateadded` to database due to faulty upsert operation!
|
|
||||||
|
|
||||||
### Problem Details
|
|
||||||
**Database Method Bug** in `core/database.py:173-182`:
|
|
||||||
- `upsert_movie_dates()` was doing UPDATE-only, not proper upsert
|
|
||||||
- If movie row didn't exist properly, UPDATE would affect 0 rows
|
|
||||||
- `dateadded` field remained NULL even after manual scan
|
|
||||||
- Webhooks found database entries but with NULL `dateadded` → fell back to current timestamp
|
|
||||||
|
|
||||||
### 🔧 **Fix Applied**
|
|
||||||
- **Location**: `core/database.py:173-186`
|
|
||||||
- **Changed**: UPDATE-only → proper `INSERT OR REPLACE` upsert
|
|
||||||
- **Result**: Manual scans now properly save `dateadded` to database
|
|
||||||
- **Webhook Impact**: Now finds existing dates correctly, no more false timestamps
|
|
||||||
|
|
||||||
### Before vs After Database Operations
|
|
||||||
**Before (BROKEN):**
|
|
||||||
```sql
|
|
||||||
UPDATE movies SET dateadded = '2025-06-15...' WHERE imdb_id = 'tt123';
|
|
||||||
-- If row doesn't exist properly: 0 rows affected, dateadded stays NULL
|
|
||||||
```
|
|
||||||
|
|
||||||
**After (FIXED):**
|
|
||||||
```sql
|
|
||||||
INSERT OR REPLACE INTO movies (...) VALUES (...);
|
|
||||||
-- Always works: Creates row OR replaces with all data including dateadded
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎯 FIXED: Webhook Date Logic Issue (September 15, 2025)
|
|
||||||
|
|
||||||
### Issue Discovery
|
|
||||||
Upgrade webhooks were incorrectly using current timestamp (`webhook:first_seen`) instead of checking Radarr for proper import dates when no database entry existed.
|
|
||||||
|
|
||||||
**Example Problem:**
|
|
||||||
- Database populated via manual scan: `2025-06-15T11:10:31-04:00` (radarr:db.history.import)
|
|
||||||
- Upgrade webhook comes in: `2025-09-15T12:04:23-04:00` (webhook:first_seen) ❌ **WRONG!**
|
|
||||||
|
|
||||||
### Root Cause
|
|
||||||
Webhook mode was bypassing the full date decision logic (`_decide_movie_dates`) and jumping straight to current timestamp as fallback.
|
|
||||||
|
|
||||||
### ✅ **Fix Applied**
|
|
||||||
- **Location**: `nfoguard.py:975-990`
|
|
||||||
- **Logic**: Webhook mode now uses same date priority as manual scans:
|
|
||||||
1. **Database first** (if exists)
|
|
||||||
2. **Full date logic** (Radarr import → release dates → fallbacks)
|
|
||||||
3. **Current timestamp** (only as absolute last resort)
|
|
||||||
|
|
||||||
### Result
|
|
||||||
Webhook upgrades now properly find and use existing Radarr import dates instead of creating false "first seen" timestamps.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔧 FIXED: Radarr Rename Event Issue (September 15, 2025)
|
|
||||||
|
|
||||||
### Issue Resolution
|
|
||||||
**COMPLETED**: Fixed missing Radarr rename event handling by adding proper event type filtering.
|
|
||||||
|
|
||||||
### ✅ **Both Sonarr & Radarr Webhook Handling** - **NOW WORKING CORRECTLY**
|
|
||||||
- **Sonarr Location**: `nfoguard.py:1429`
|
|
||||||
- **Radarr Location**: `nfoguard.py:1479` **(FIXED)**
|
|
||||||
- **Event Filtering**: `if event_type not in ["Download", "Upgrade", "Rename"]:`
|
|
||||||
- **✅ STATUS**: **Rename events ARE processed for both systems**
|
|
||||||
- **✅ BEHAVIOR**: Both systems now update NFO files with proper date metadata on rename
|
|
||||||
|
|
||||||
### What Was Fixed
|
|
||||||
1. **Added Event Type Filtering**: Radarr webhooks now explicitly filter for supported events
|
|
||||||
2. **Consistent Processing**: Both Sonarr and Radarr now follow identical webhook workflows
|
|
||||||
3. **Proper Rename Handling**: Rename events trigger full NFO processing pipeline
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📋 NFOGuard Webhook Workflow Documentation
|
|
||||||
|
|
||||||
### 🎬 **Radarr Webhook Workflow**
|
|
||||||
1. **Import Event**:
|
|
||||||
- Check database - if not seen before, current timestamp = truth of first import
|
|
||||||
- Update movie.nfo with proper date metadata
|
|
||||||
|
|
||||||
2. **Rename Event**:
|
|
||||||
- Check database - if we have the date, use existing date
|
|
||||||
- Update movie.nfo file as always (preserving original import date)
|
|
||||||
|
|
||||||
3. **Rename Event (No Database Entry)**:
|
|
||||||
- Check Radarr database for earliest import date
|
|
||||||
- If found, use earliest import date → update database + movie.nfo
|
|
||||||
- If no import date found, use digital/theatrical release date (fallback logic)
|
|
||||||
|
|
||||||
### 📺 **Sonarr Webhook Workflow**
|
|
||||||
1. **Import Event**:
|
|
||||||
- Check database - if not seen before, current timestamp = truth of first import
|
|
||||||
- Update episode.nfo with proper date metadata
|
|
||||||
|
|
||||||
2. **Rename Event**:
|
|
||||||
- Check database - if we have the date, use existing date
|
|
||||||
- Update episode.nfo file as always (preserving original import date)
|
|
||||||
|
|
||||||
3. **Rename Event (No Database Entry)**:
|
|
||||||
- Check Sonarr API for earliest import date
|
|
||||||
- If found, use earliest import date → update database + episode.nfo
|
|
||||||
- If no import date found, use air date (fallback logic)
|
|
||||||
|
|
||||||
### 🔄 **Key Processing Logic**
|
|
||||||
- **Database First**: Always check NFOGuard database for existing dates
|
|
||||||
- **Import Truth**: Webhook import events establish "source of truth" timestamps
|
|
||||||
- **Date Preservation**: Rename events preserve original import dates, never overwrite
|
|
||||||
- **Smart Fallbacks**: API queries → Release dates → Air dates → File dates (configurable)
|
|
||||||
- **Batch Processing**: All webhooks go through batching system with configurable delays
|
|
||||||
Binary file not shown.
Reference in New Issue
Block a user