nfoguard/README.md

# NFOGuard


*** Alpha in progress things are volatile use at your own risk ***

**NFOGuard** is a lightweight webhook service that locks in the *true import date* of your movies and TV episodes, ensuring that upgrades, renames, or reimports never bubble old media up as “new” in Emby, Jellyfin, or Plex.  

It integrates with **Sonarr** and **Radarr**, listens for import/rename/upgrade events, and automatically manages `.nfo` files and filesystem timestamps to keep your library chronology consistent.

---

## ✨ Features

- **Import Date Locking**  
  Captures the original date a movie or episode was imported and preserves it across upgrades.

- **NFO Management**  
  Creates and updates `.nfo` files for movies and TV episodes with consistent `<dateadded>` fields.  
  Optionally adds `<lockdata>` to prevent metadata drift from scrapers.

- **Filesystem Time Fixing**  
  Updates file and directory modification times (`mtime`) to match the preserved import date.

- **Batching Support**  
  Groups multiple webhook events (e.g. whole-season downloads) to avoid thrashing.

- **Database Backing**  
  Uses SQLite to remember previously-seen imports, so dates persist across restarts.

- **Manual Scans**  
  Exposes endpoints to rescan and reconcile TV or movie libraries on demand.

---

## 🏗 How It Works

1. **Webhooks**  
   - Add NFOGuard as a webhook in Sonarr and Radarr.  
   - Supported event types: `Download`, `Upgrade`, `Rename`.

2. **Processing**  
   - Maps Sonarr/Radarr paths to container paths.  
   - Looks up IMDb IDs from folder names or payloads.  
   - Writes `.nfo` files and updates file/dir mtimes.  
   - Records everything in `media_dates.db`.

3. **Playback Apps**  
   - Emby/Jellyfin/Plex see consistent `dateadded` values.  
   - Upgrades don’t bubble to the top of “Recently Added.”

---

## 🚀 Quick Start (Docker Compose)

```bash
# 1. Copy configuration templates
cp .env.template .env
cp .env.secrets.template .env.secrets

# 2. Edit your configuration files
# .env - paths and preferences (safe to share)
# .env.secrets - API keys and passwords (never commit)

# 3. Start NFOGuard
docker-compose up -d
```

**Docker Compose (Recommended)**:
```yaml
version: '3.8'
services:
  nfoguard:
    image: ghcr.io/your-org/nfoguard:latest
    container_name: nfoguard
    ports:
      - "8080:8080"
    volumes:
      - /mnt/unionfs/Media:/media:rw
      - ./data:/app/data
      - ./.env:/app/.env:ro              # Main configuration
      - ./.env.secrets:/app/.env.secrets:ro  # Secure API keys
    restart: unless-stopped
```

**Docker Run (Alternative)**:
```bash
docker run -d \
  --name=NFOGuard \
  -p 8080:8080 \
  -v /mnt/unionfs/Media:/media:rw \
  -v /opt/NFOGuard/data:/app/data \
  -v /opt/NFOGuard/.env:/app/.env:ro \
  -v /opt/NFOGuard/.env.secrets:/app/.env.secrets:ro \
  ghcr.io/your-org/NFOGuard:latest
```

## 🔧 Configuration Files

NFOGuard uses a **secure two-file configuration system**:

### **`.env` - Main Configuration** (safe to share)
```bash
# Media paths
MOVIE_PATHS=/media/Movies/movies,/media/Movies/movies6
TV_PATHS=/media/TV/tv,/media/TV/tv6

# Release date priority (digital, physical, theatrical)
RELEASE_DATE_PRIORITY=digital,physical,theatrical
PREFER_RELEASE_DATES_OVER_FILE_DATES=true
ALLOW_FILE_DATE_FALLBACK=false

# Smart date validation: prefer theatrical over unreasonably late dates
ENABLE_SMART_DATE_VALIDATION=true
MAX_RELEASE_DATE_GAP_YEARS=10

# Database connection
RADARR_DB_HOST=radarr-postgres
RADARR_DB_PORT=5432
RADARR_DB_NAME=radarr
RADARR_DB_USER=postgres
```

### **`.env.secrets` - Sensitive Data** (never commit)
```bash
# Database password
RADARR_DB_PASSWORD=your_actual_password

# API keys
TMDB_API_KEY=your_tmdb_api_key
RADARR_API_KEY=your_radarr_api_key
SONARR_API_KEY=your_sonarr_api_key
```

**Security Features**:
- 🔐 **API key masking** in logs automatically
- 🚫 **Git protection** - secrets never committed
- 🛠️ **Easy debugging** - main .env safe to share

🔌 API Endpoints

### Webhook Endpoints
```bash
# Radarr webhook (automatic processing)
POST /webhook/radarr

# Sonarr webhook (automatic processing)  
POST /webhook/sonarr
```

### Manual Processing
```bash
# Manual scan all media
curl -X POST "http://localhost:8080/manual/scan?scan_type=both"

# Manual scan TV only
curl -X POST "http://localhost:8080/manual/scan?scan_type=tv"

# Manual scan movies only
curl -X POST "http://localhost:8080/manual/scan?scan_type=movies"

# Manual scan specific path
curl -X POST "http://localhost:8080/manual/scan?path=/media/movies"

# Bulk update all movies from Radarr database
curl -X POST "http://localhost:8080/bulk/update"
```

### Testing & Validation
```bash
# Test database connections
curl -X POST "http://localhost:8080/test/bulk-update"

# Test movie directory scanning
curl -X POST "http://localhost:8080/test/movie-scan"

# System health check
curl "http://localhost:8080/health"

# Database statistics
curl "http://localhost:8080/stats"

# Batch processing queue status
curl "http://localhost:8080/batch/status"
```

### Debugging & Analysis
```bash
# Debug specific movie import date detection
curl "http://localhost:8080/debug/movie/tt1674782"

# Show complete import history analysis
curl "http://localhost:8080/debug/movie/tt1674782/history"

# Show date priority logic and available sources
curl "http://localhost:8080/debug/movie/tt1674782/priority"

# Debug TMDB API lookup step by step
curl "http://localhost:8080/debug/tmdb/tt1674782"
```

### Response Examples

**Health Check**:
```json
{
  "status": "healthy",
  "version": "0.5.1", 
  "database_status": "healthy",
  "radarr_database": {"status": "healthy", "movies": 1500}
}
```

**Movie Debug**:
```json
{
  "detected_import_date": "2025-07-08T03:30:04+00:00",
  "import_source": "radarr:history.import",
  "movie_title": "Movie Name"
}
```

**Priority Logic Debug**:
```json
{
  "movie_priority": "import_then_digital",
  "release_date_priority": ["digital", "physical", "theatrical"],
  "priority_explanation": "1st: Radarr import history, 2nd: Release dates (digital → physical → theatrical), 3rd: file mtime",
  "date_sources": {
    "radarr_import": {"date": "2025-08-29T12:14:28+00:00", "source": "radarr:db.file.dateAdded"},
    "digital_release": {"date": "1996-05-03T00:00:00+00:00", "source": "tmdb:theatrical"}
  },
  "file_date_detected": true,
  "would_prefer_digital": true,
  "selected_date": "1996-05-03T00:00:00+00:00",
  "selected_source": "tmdb:theatrical (preferred over file date)"
}
```

## 🎯 Release Date Priority System

NFOGuard uses a **smart fallback system** when Radarr has no valid import history:

### 1. **Digital Release** (streaming/VOD)
- Netflix, Amazon Prime, iTunes, etc.
- **TMDB Source**: Release type 4
- **Typical**: 45-90 days after theatrical

### 2. **Physical Release** (DVD/Blu-ray) 
- Physical media release dates
- **TMDB Source**: Release type 5  
- **Typical**: 90-120 days after theatrical

### 3. **Theatrical Release** (cinema)
- Original movie theater release
- **TMDB Source**: Release type 3
- **Most Authoritative**: Always used when no digital/physical exists

## 🧠 Smart Date Validation

**NEW**: Automatically detects unreasonable date gaps and chooses the most logical release date.

### How It Works
- **Compares release dates**: Checks if digital/physical releases are unreasonably far from theatrical
- **Automatic fallback**: If gap exceeds threshold, prefers theatrical date instead
- **Configurable threshold**: `MAX_RELEASE_DATE_GAP_YEARS=10` (default: 10 years)

### Real Example: "The Craft (1996)"
```bash
# Without smart validation:
Physical Release: 2022-05-17 (26 years after theatrical!)
Selected: 2022 ❌

# With smart validation:
Theatrical: 1996-05-03
Physical: 2022-05-17 (26 year gap > 10 year limit)
Selected: 1996 theatrical ✅ (smart fallback)
```

**Configuration**:
```bash
# Enable/disable smart validation
ENABLE_SMART_DATE_VALIDATION=true

# Maximum reasonable gap in years
MAX_RELEASE_DATE_GAP_YEARS=10
```

### Configuration Examples

**For "The Craft (1996)"** (predates digital):
- ❌ Digital: Not available (movie too old)
- ❌ Physical: DVD from 2000 (too late) 
- ✅ **Theatrical: 1996-05-03** ← **Selected**

**For "Top Gun Maverick (2022)"** (modern movie):
- ✅ **Digital: 2022-08-23** ← **Selected** (your priority)
- ⏭️ Physical: 2022-10-31 (skipped due to priority)
- ⏭️ Theatrical: 2022-05-27 (skipped due to priority)

## 🐛 Troubleshooting

### Manual Imports Show File Dates Instead of Release Dates

**Problem**: Movies manually imported show recent file dates like `2025-08-29T12:14:28+00:00`

**Solution**:
1. Enable TMDB integration and configure fallback priority:
   ```bash
   # Add to your .env file
   TMDB_API_KEY=your_tmdb_api_key
   PREFER_RELEASE_DATES_OVER_FILE_DATES=true
   RELEASE_DATE_PRIORITY=digital,physical,theatrical
   ```

2. Test the priority logic:
   ```bash
   curl "http://localhost:8080/debug/movie/tt0115963/priority"
   ```

3. Should show release date selection based on your priority order

### "0 Movies Processed" During Manual Scan

**Problem**: Manual scan reports processing 0 movies

**Solution**:
1. Test directory scanning:
   ```bash
   curl -X POST "http://localhost:8080/test/movie-scan"
   ```

2. Verify paths in your `.env` file match your actual structure:
   ```bash
   # Your structure: /media/Movies/movies/
   MOVIE_PATHS=/media/Movies/movies,/media/Movies/movies6
   ```

3. Ensure movie directories have IMDb tags:
   ```
   Movie Name [imdb-tt1234567] (2023)/
   ```

### Database Connection Issues

**Problem**: "Radarr database connection failed"

**Solution**:
1. Test database connectivity:
   ```bash
   curl -X POST "http://localhost:8080/test/bulk-update"
   ```

2. Verify database credentials in `.env`:
   ```bash
   RADARR_DB_HOST=radarr-postgres
   RADARR_DB_PASSWORD=your_actual_password
   ```

### Release Dates Not Found

**Problem**: `"external_apis": {"tmdb_enabled": false}` or no release dates detected

**Solution**:
1. Add TMDB API key and configure fallback order:
   ```bash
   # Add to your .env
   TMDB_API_KEY=your_api_key
   RELEASE_DATE_PRIORITY=digital,physical,theatrical
   ```
2. Restart container: `docker-compose restart`
3. Test all release types: `curl "http://localhost:8080/debug/tmdb/tt0115963"`
4. Verify priority logic: `curl "http://localhost:8080/debug/movie/tt0115963/priority"`

### Customizing Release Date Priority

**Default Order**: `digital,physical,theatrical` (your preference)
**Alternative Orders**:
```bash
# Prefer theatrical dates for older movies
RELEASE_DATE_PRIORITY=theatrical,digital,physical

# Physical media collector preference  
RELEASE_DATE_PRIORITY=physical,digital,theatrical

# Digital-first modern preference
RELEASE_DATE_PRIORITY=digital,theatrical,physical
```

### Disabling File Date Fallbacks

**Problem**: Don't want file modification dates used at all

**Solution**: Disable file date fallbacks completely
```bash
# Add to your .env
ALLOW_FILE_DATE_FALLBACK=false
```

**Result**: 
- ✅ Movies with release dates get processed normally
- ⚠️ Movies with NO release dates get skipped (no NFO created)  
- 🔇 No more "Using file dateAdded as fallback" warnings

## 📖 Example Workflow
You add The Blacklist in Sonarr.

Sonarr downloads S01E01 → NFOGuard logs the import date in DB and .nfo.

Months later, a better-quality upgrade replaces the file.

NFOGuard updates the .nfo and mtime, but keeps the original import date.

Emby/Jellyfin/Plex see the file as unchanged in chronology.

Result: no more "old shows" showing up in "Recently Added."

🔍 Logging & Debugging
# Enable verbose logging
DEBUG=true

# Check container logs
docker logs sonarr-nfo-cache

# Debug specific movie import detection
curl http://localhost:8080/debug/movie/tt1674782

# Manual scan with verbose output
curl -X POST "http://localhost:8080/manual/scan?scan_type=movies"

# File logs (inside container)
/app/data/logs/nfoguard.log

⚙️ Environment Variables (v0.2.1+)
Variable	Default	Description
RADARR_ROOT_FOLDERS	(required)	What Radarr sees: /mnt/unionfs/Media/Movies/movies
SONARR_ROOT_FOLDERS	(required)	What Sonarr sees: /mnt/unionfs/Media/TV/tv  
DOWNLOAD_PATH_INDICATORS	(see example)	Paths that indicate downloads vs existing files
MOVIE_PRIORITY	import_then_digital	Strategy: import_then_digital or digital_then_import
MOVIE_POLL_MODE	always	When to query APIs: always, if_missing, never
MOVIE_DATE_UPDATE_MODE	backfill_only	Update behavior: backfill_only or overwrite

📜 License
MIT License – do what you want, just don’t blame us if your timestamps go timey-wimey.