nfoguard/README.md

# NFOGuard

**NFOGuard** is a lightweight webhook service that locks in the *true import date* of your movies and TV episodes, ensuring that upgrades, renames, or reimports never bubble old media up as "new" in Emby, Jellyfin, or Plex.  

It integrates with **Sonarr** and **Radarr**, listens for import/rename/upgrade events, and automatically manages `.nfo` files and filesystem timestamps to keep your library chronology consistent.

## Companion Plugin

For **Emby users**, check out the companion [NFOGuard Emby Plugin](https://github.com/jskala/NFOGuard-Emby-Plugin) that provides real-time date synchronization within Emby itself.

---

## ✨ Features

- **Import Date Locking**  
  Captures the original date a movie or episode was imported and preserves it across upgrades.

- **Enhanced NFO Management**  
  Creates and updates `.nfo` files for movies and TV episodes with consistent `<dateadded>` fields.  
  **NEW**: Full metadata integration from Sonarr API (episode titles, plots, ratings, runtime).  
  Optionally adds `<lockdata>` to prevent metadata drift from scrapers.  
  **NEW**: Timestamps all NFO files with creation/update dates.

- **Filesystem Time Fixing**  
  Updates file and directory modification times (`mtime`) to match the preserved import date.

- **Batching Support**  
  Groups multiple webhook events (e.g. whole-season downloads) to avoid thrashing.

- **Database Backing**  
  Uses SQLite to remember previously-seen imports, so dates persist across restarts.

- **Manual Scans**  
  Exposes endpoints to rescan and reconcile TV or movie libraries on demand.

---

## 🏗 How It Works

1. **Webhooks**  
   - Add NFOGuard as a webhook in Sonarr and Radarr.  
   - Supported event types: `Download`, `Upgrade`, `Rename`.

2. **Processing**  
   - Maps Sonarr/Radarr paths to container paths.  
   - Looks up IMDb IDs from folder names or payloads.  
   - Writes `.nfo` files and updates file/dir mtimes.  
   - Records everything in `media_dates.db`.

3. **Playback Apps**  
   - Emby/Jellyfin/Plex see consistent `dateadded` values.  
   - Upgrades don’t bubble to the top of “Recently Added.”

---

## 🚀 Quick Start (Docker Compose)

```bash
# 1. Copy configuration templates
cp .env.template .env
cp .env.secrets.template .env.secrets

# 2. Edit your configuration files
# .env - paths and preferences (safe to share)
# .env.secrets - API keys and passwords (never commit)

# 3. Start NFOGuard
docker-compose up -d
```

**Docker Compose (Recommended)**:
```yaml
version: '3.8'
services:
  nfoguard:
    image: ghcr.io/your-org/nfoguard:latest
    container_name: nfoguard
    ports:
      - "8080:8080"
    volumes:
      - /mnt/unionfs/Media:/media:rw
      - ./data:/app/data
      - ./.env:/app/.env:ro              # Main configuration
      - ./.env.secrets:/app/.env.secrets:ro  # Secure API keys
    restart: unless-stopped
```

**Docker Run (Alternative)**:
```bash
docker run -d \
  --name=NFOGuard \
  -p 8080:8080 \
  -v /mnt/unionfs/Media:/media:rw \
  -v /opt/NFOGuard/data:/app/data \
  -v /opt/NFOGuard/.env:/app/.env:ro \
  -v /opt/NFOGuard/.env.secrets:/app/.env.secrets:ro \
  ghcr.io/your-org/NFOGuard:latest
```

## 🔧 Configuration Files

NFOGuard uses a **secure two-file configuration system**:

### **`.env` - Main Configuration** (safe to share)
```bash
# Media paths
MOVIE_PATHS=/media/Movies/movies,/media/Movies/movies6
TV_PATHS=/media/TV/tv,/media/TV/tv6

# Release date priority (digital, physical, theatrical)
RELEASE_DATE_PRIORITY=digital,physical,theatrical
PREFER_RELEASE_DATES_OVER_FILE_DATES=true
ALLOW_FILE_DATE_FALLBACK=false

# Smart date validation: prefer theatrical over unreasonably late dates
ENABLE_SMART_DATE_VALIDATION=true
MAX_RELEASE_DATE_GAP_YEARS=10

# TV webhook processing mode (v0.6.0+)
# targeted = Only process episodes in webhook (efficient)
# series = Process entire series directory (comprehensive)
TV_WEBHOOK_PROCESSING_MODE=targeted

# Database connection
RADARR_DB_HOST=radarr-postgres
RADARR_DB_PORT=5432
RADARR_DB_NAME=radarr
RADARR_DB_USER=postgres
```

### **`.env.secrets` - Sensitive Data** (never commit)
```bash
# Database password
RADARR_DB_PASSWORD=your_actual_password

# API keys
TMDB_API_KEY=your_tmdb_api_key
TVDB_API_KEY=your_tvdb_api_key
RADARR_API_KEY=your_radarr_api_key
SONARR_API_KEY=your_sonarr_api_key
```

**Security Features**:
- 🔐 **API key masking** in logs automatically
- 🚫 **Git protection** - secrets never committed
- 🛠️ **Easy debugging** - main .env safe to share

🔌 API Endpoints

### Webhook Endpoints
```bash
# Radarr webhook (automatic processing)
POST /webhook/radarr

# Sonarr webhook (automatic processing)  
POST /webhook/sonarr
```

### Manual Processing
```bash
# Manual scan all media
curl -X POST "http://localhost:8080/manual/scan?scan_type=both"

# Manual scan TV only
curl -X POST "http://localhost:8080/manual/scan?scan_type=tv"

# Manual scan movies only
curl -X POST "http://localhost:8080/manual/scan?scan_type=movies"

# Manual scan specific path
curl -X POST "http://localhost:8080/manual/scan?path=/media/movies"

# Bulk update all movies from Radarr database
curl -X POST "http://localhost:8080/bulk/update"
```

### TV Show Processing (NEW in v0.6.0+)
```bash
# Process a specific TV season (URL-safe)
curl -X POST "http://localhost:8080/tv/scan-season" \
  -H "Content-Type: application/json" \
  -d '{
    "series_path": "/media/TV/tv/Chicago Fire (2012) [imdb-tt2261391]",
    "season_name": "Season 13"
  }'

# Process a specific TV episode (URL-safe)
curl -X POST "http://localhost:8080/tv/scan-episode" \
  -H "Content-Type: application/json" \
  -d '{
    "series_path": "/media/TV/tv/Chicago Fire (2012) [imdb-tt2261391]",
    "season_name": "Season 13", 
    "episode_name": "Chicago Fire (2012)-S13E21-The Bad Guy[WEBDL-1080p][AAC2.0][h264].mkv"
  }'

# Process single season via manual scan (URL encoding required for spaces)
curl -X POST "http://localhost:8080/manual/scan?path=/media/TV/tv/Chicago%20Fire%20(2012)%20%5Bimdb-tt2261391%5D/Season%2013"

# Process single episode via manual scan (URL encoding required for spaces)
curl -X POST "http://localhost:8080/manual/scan?path=/media/TV/tv/Chicago%20Fire%20(2012)%20%5Bimdb-tt2261391%5D/Season%2013/episode.mkv"
```

### Testing & Validation
```bash
# Test database connections
curl -X POST "http://localhost:8080/test/bulk-update"

# Test movie directory scanning
curl -X POST "http://localhost:8080/test/movie-scan"

# System health check
curl "http://localhost:8080/health"

# Database statistics
curl "http://localhost:8080/stats"

# Batch processing queue status
curl "http://localhost:8080/batch/status"
```

### Debugging & Analysis
```bash
# Debug specific movie import date detection
curl "http://localhost:8080/debug/movie/tt1674782"

# Show complete import history analysis
curl "http://localhost:8080/debug/movie/tt1674782/history"

# Show date priority logic and available sources
curl "http://localhost:8080/debug/movie/tt1674782/priority"

# Debug TMDB API lookup step by step
curl "http://localhost:8080/debug/tmdb/tt1674782"
```

### Response Examples

**Health Check**:
```json
{
  "status": "healthy",
  "version": "0.6.0", 
  "database_status": "healthy",
  "radarr_database": {"status": "healthy", "movies": 1500}
}
```

**Movie Debug**:
```json
{
  "detected_import_date": "2025-07-08T03:30:04+00:00",
  "import_source": "radarr:history.import",
  "movie_title": "Movie Name"
}
```

**Priority Logic Debug**:
```json
{
  "movie_priority": "import_then_digital",
  "release_date_priority": ["digital", "physical", "theatrical"],
  "priority_explanation": "1st: Radarr import history, 2nd: Release dates (digital → physical → theatrical), 3rd: file mtime",
  "date_sources": {
    "radarr_import": {"date": "2025-08-29T12:14:28+00:00", "source": "radarr:db.file.dateAdded"},
    "digital_release": {"date": "1996-05-03T00:00:00+00:00", "source": "tmdb:theatrical"}
  },
  "file_date_detected": true,
  "would_prefer_digital": true,
  "selected_date": "1996-05-03T00:00:00+00:00",
  "selected_source": "tmdb:theatrical (preferred over file date)"
}
```

**TV Season Processing**:
```json
{
  "status": "started",
  "message": "Season scan started for Season 13"
}
```

**TV Episode Processing**:
```json
{
  "status": "started", 
  "message": "Episode scan started for Chicago Fire (2012)-S13E21-The Bad Guy[WEBDL-1080p][AAC2.0][h264].mkv"
}
```

## 🎯 Date Selection Priority System

NFOGuard uses a **comprehensive fallback hierarchy** to find the best possible date for each movie:

### Movie Priority: `import_then_digital` (Default)

**1. Radarr Import History** *(Highest Priority)*
- `radarr:db.history.import` - Real download/import events from Radarr database
- `radarr:db.history.grab` - Fallback to grab events if no import events exist
- **Skip if**: First event is rename (indicates upgrades, not original import)

**2. TMDB Release Dates** *(When import unavailable/unreliable)*
- `tmdb:premiere` - Type 1: World premieres, film festivals  
- `tmdb:limited` - Type 2: Limited theatrical releases
- `tmdb:theatrical` - Type 3: Wide theatrical releases
- `tmdb:digital` - Type 4: Digital/streaming releases (Netflix, iTunes, etc.)
- `tmdb:physical` - Type 5: Physical media (DVD, Blu-ray)
- `tmdb:tv` - Type 6: TV broadcasts, TV movie premieres
- **Priority Order**: Uses `RELEASE_DATE_PRIORITY=digital,physical,theatrical`

**3. Radarr NFO Premiered Date** *(Radarr's own research)*
- `radarr:nfo.premiered` - Extracts `<premiered>YYYY-MM-DD</premiered>` from existing movie.nfo
- **Source**: Radarr's metadata providers (TMDB, IMDb, etc.)

**4. File Modification Time** *(Last Resort)*
- `file:mtime` - Earliest video file modification time in directory
- **Only if**: `ALLOW_FILE_DATE_FALLBACK=true`

### Movie Priority: `digital_then_import`

Same hierarchy, but **TMDB Release Dates** are tried **before** Radarr Import History.

## 🧠 Smart Date Validation

**NEW**: Automatically detects unreasonable date gaps and chooses the most logical release date.

### How It Works
- **Compares release dates**: Checks if digital/physical releases are unreasonably far from theatrical
- **Automatic fallback**: If gap exceeds threshold, prefers theatrical date instead
- **Configurable threshold**: `MAX_RELEASE_DATE_GAP_YEARS=10` (default: 10 years)

### Real Example: "The Craft (1996)"
```bash
# Without smart validation:
Physical Release: 2022-05-17 (26 years after theatrical!)
Selected: 2022 ❌

# With smart validation:
Theatrical: 1996-05-03
Physical: 2022-05-17 (26 year gap > 10 year limit)
Selected: 1996 theatrical ✅ (smart fallback)
```

**Configuration**:
```bash
# Enable/disable smart validation
ENABLE_SMART_DATE_VALIDATION=true

# Maximum reasonable gap in years
MAX_RELEASE_DATE_GAP_YEARS=10
```

### Configuration Examples

**For "The Craft (1996)"** (predates digital):
- ❌ Digital: Not available (movie too old)
- ❌ Physical: DVD from 2000 (too late) 
- ✅ **Theatrical: 1996-05-03** ← **Selected**

**For "Top Gun Maverick (2022)"** (modern movie):
- ✅ **Digital: 2022-08-23** ← **Selected** (your priority)
- ⏭️ Physical: 2022-10-31 (skipped due to priority)
- ⏭️ Theatrical: 2022-05-27 (skipped due to priority)

## 🐛 Troubleshooting

### Manual Imports Show File Dates Instead of Release Dates

**Problem**: Movies manually imported show recent file dates like `2025-08-29T12:14:28+00:00`

**Solution**:
1. Enable TMDB integration and configure fallback priority:
   ```bash
   # Add to your .env file
   TMDB_API_KEY=your_tmdb_api_key
   PREFER_RELEASE_DATES_OVER_FILE_DATES=true
   RELEASE_DATE_PRIORITY=digital,physical,theatrical
   ```

2. Test the priority logic:
   ```bash
   curl "http://localhost:8080/debug/movie/tt0115963/priority"
   ```

3. Should show release date selection based on your priority order

### "0 Movies Processed" During Manual Scan

**Problem**: Manual scan reports processing 0 movies

**Solution**:
1. Test directory scanning:
   ```bash
   curl -X POST "http://localhost:8080/test/movie-scan"
   ```

2. Verify paths in your `.env` file match your actual structure:
   ```bash
   # Your structure: /media/Movies/movies/
   MOVIE_PATHS=/media/Movies/movies,/media/Movies/movies6
   ```

3. Ensure movie directories have IMDb tags:
   ```
   Movie Name [imdb-tt1234567] (2023)/
   ```

### Database Connection Issues

**Problem**: "Radarr database connection failed"

**Solution**:
1. Test database connectivity:
   ```bash
   curl -X POST "http://localhost:8080/test/bulk-update"
   ```

2. Verify database credentials in `.env`:
   ```bash
   RADARR_DB_HOST=radarr-postgres
   RADARR_DB_PASSWORD=your_actual_password
   ```

### Release Dates Not Found

**Problem**: `"external_apis": {"tmdb_enabled": false}` or no release dates detected

**Solution**:
1. Add TMDB API key and configure fallback order:
   ```bash
   # Add to your .env
   TMDB_API_KEY=your_api_key
   RELEASE_DATE_PRIORITY=digital,physical,theatrical
   ```
2. Restart container: `docker-compose restart`
3. Test all release types: `curl "http://localhost:8080/debug/tmdb/tt0115963"`
4. Verify priority logic: `curl "http://localhost:8080/debug/movie/tt0115963/priority"`

### Customizing Release Date Priority

**Default Order**: `digital,physical,theatrical` (your preference)
**Alternative Orders**:
```bash
# Prefer theatrical dates for older movies
RELEASE_DATE_PRIORITY=theatrical,digital,physical

# Physical media collector preference  
RELEASE_DATE_PRIORITY=physical,digital,theatrical

# Digital-first modern preference
RELEASE_DATE_PRIORITY=digital,theatrical,physical
```

### Disabling File Date Fallbacks

**Problem**: Don't want file modification dates used at all

**Solution**: Disable file date fallbacks completely
```bash
# Add to your .env
ALLOW_FILE_DATE_FALLBACK=false
```

**Result**: 
- ✅ Movies with release dates get processed normally
- ⚠️ Movies with NO release dates get skipped (no NFO created)  
- 🔇 No more "Using file dateAdded as fallback" warnings

## 📱 Media Server Compatibility

### Movies
- **Emby**: ✅ Full support - reads NFO `<dateadded>` fields
- **Jellyfin**: ✅ Full support - reads NFO `<dateadded>` fields  
- **Plex**: ✅ Full support - reads NFO `<dateadded>` fields

### TV Episodes
- **Emby**: ⚠️ **Partial support** - reads metadata (titles, plots, ratings) but **ignores all NFO date fields**
- **Jellyfin**: ✅ Full support - reads NFO `<dateadded>` fields
- **Plex**: ✅ Full support - reads NFO `<dateadded>` fields

> **Note**: Emby has an architectural limitation where TV episodes always use filesystem dates instead of NFO dates. This is a known Emby behavior that cannot be worked around. NFOGuard still provides valuable enhanced metadata for TV episodes in Emby, just not corrected dates.

---

## 🔑 API Keys Configuration

NFOGuard integrates with multiple external APIs to provide enhanced metadata and release date information.

### Required API Keys

| Service | Environment Variable | Purpose | How to Get |
|---------|---------------------|---------|------------|
| **TMDB** | `TMDB_API_KEY` | Movie release dates, metadata fallbacks | Free at [themoviedb.org](https://www.themoviedb.org/settings/api) |
| **TVDB** | `TVDB_API_KEY` | TV show metadata, Emby compatibility | Free at [thetvdb.com](https://thetvdb.com/api-information) |
| **Radarr** | `RADARR_API_KEY` | Movie import history, database access | Found in Radarr Settings → General |
| **Sonarr** | `SONARR_API_KEY` | TV episode import history | Found in Sonarr Settings → General |

### Configuration Example

Add to your `.env.secrets` file:

```bash
# External API keys for enhanced metadata
TMDB_API_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
TVDB_API_KEY=your-tvdb-api-key-here

# *arr application API keys  
RADARR_API_KEY=a1b2c3d4e5f6...
SONARR_API_KEY=f6e5d4c3b2a1...
```

### API Key Warnings

If you see these warnings in logs, add the corresponding API key:

```bash
[WARNING] TMDB API key not configured - release date fallbacks disabled
[WARNING] TVDB API key not configured, skipping TVDB ID lookup
```

**Note**: TMDB and TVDB keys are **optional** but highly recommended for:
- ✅ Better release date accuracy
- ✅ Enhanced TV show metadata  
- ✅ Improved Emby/Plex compatibility

---

## 📖 Example Workflow
You add The Blacklist in Sonarr.

Sonarr downloads S01E01 → NFOGuard logs the import date in DB and .nfo.

Months later, a better-quality upgrade replaces the file.

NFOGuard updates the .nfo and mtime, but keeps the original import date.

Emby/Jellyfin/Plex see the file as unchanged in chronology.

Result: no more "old shows" showing up in "Recently Added."

## 📺 Enhanced NFO Generation (NEW)

NFOGuard now creates **full-featured NFO files** with rich metadata from Sonarr:

### Before (Basic Format):
```xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<episodedetails>
  <season>13</season>
  <episode>2</episode>
  <aired>2024-10-03</aired>
  <dateadded>2025-02-19T23:10:35+00:00</dateadded>
  <premiered>2025-02-19</premiered>
  <lockdata>true</lockdata>
  <!-- source: TV Episode; sonarr:history.import -->
  <!-- managed by NFOGuard -->
</episodedetails>
```

### After (Enhanced Format):
```xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<episodedetails>
  <season>13</season>
  <episode>2</episode>
  <title>Ride the Blade</title>
  <plot>Firehouse 51 tackles a fire at a local restaurant. Boden has an emotional conversation with his stepson.</plot>
  <runtime>42</runtime>
  <rating>8.2</rating>
  <votes>1543</votes>
  <aired>2024-10-03</aired>
  <dateadded>2025-02-19T23:10:35+00:00</dateadded>
  <premiered>2025-02-19</premiered>
  <lockdata>true</lockdata>
  <!-- source: TV Episode; sonarr:history.import -->
  <!-- managed by NFOGuard at 2025-09-11T13:45:22+00:00 -->
</episodedetails>
```

**Enhanced features:**
- ✅ **Episode titles** from Sonarr API
- ✅ **Plot summaries** for better library browsing  
- ✅ **Runtime** and **ratings** data
- ✅ **Timestamps** showing when NFOGuard processed the file
- ✅ **Smart URL-safe endpoints** for processing individual seasons/episodes
- ✅ **Configurable processing modes** for webhook efficiency

## ⚙️ TV Webhook Processing Modes (v0.6.0+)

NFOGuard offers two processing modes for TV show webhooks:

### **Targeted Mode** (Default - `targeted`)
**Best for**: Efficient processing, minimal file operations
```bash
TV_WEBHOOK_PROCESSING_MODE=targeted
```
**Behavior**: Only processes episodes mentioned in the webhook
- ✅ **1 episode downloaded** → **1 episode processed**  
- ✅ **Minimal file operations** - only touches new episode
- ✅ **Reduced notifications** to media servers
- ✅ **Faster processing** for single episode downloads

**Example**: Download S41E07 → Only S41E07.nfo gets updated

### **Series Mode** (`series`)
**Best for**: Comprehensive updates, database synchronization
```bash
TV_WEBHOOK_PROCESSING_MODE=series  
```
**Behavior**: Processes entire TV series directory
- 📺 **1 episode downloaded** → **All series episodes processed**
- 🔄 **Complete sync** - ensures all episodes stay current  
- 📋 **Database consistency** - all episode dates updated
- 🛠️ **Legacy behavior** - matches previous versions

**Example**: Download S41E07 → All 7 episodes in season get updated

### **Smart Fallback**
- If webhook contains no episode data → automatically uses series mode
- If targeted mode fails → falls back to series processing  
- Configuration validated at startup with helpful error messages

🔍 Logging & Debugging
# Enable verbose logging
DEBUG=true

# Check container logs
docker logs sonarr-nfo-cache

# Debug specific movie import detection
curl http://localhost:8080/debug/movie/tt1674782

# Manual scan with verbose output
curl -X POST "http://localhost:8080/manual/scan?scan_type=movies"

# File logs (inside container)
/app/data/logs/nfoguard.log

⚙️ Environment Variables (v0.2.1+)
Variable	Default	Description
RADARR_ROOT_FOLDERS	(required)	What Radarr sees: /mnt/unionfs/Media/Movies/movies
SONARR_ROOT_FOLDERS	(required)	What Sonarr sees: /mnt/unionfs/Media/TV/tv  
DOWNLOAD_PATH_INDICATORS	(see example)	Paths that indicate downloads vs existing files
MOVIE_PRIORITY	import_then_digital	Strategy: import_then_digital or digital_then_import
MOVIE_POLL_MODE	always	When to query APIs: always, if_missing, never
MOVIE_DATE_UPDATE_MODE	backfill_only	Update behavior: backfill_only or overwrite

📜 License
MIT License – do what you want, just don’t blame us if your timestamps go timey-wimey.