dev #62

Merged
sbcrumb merged 56 commits from dev into main 2025-10-26 15:45:15 -04:00
22 changed files with 3114 additions and 518 deletions
+5
View File
@@ -138,6 +138,11 @@ BATCH_DELAY=5.0
# Maximum concurrent series processing
MAX_CONCURRENT_SERIES=3
# Sequential processing delay for bulk downloads (seconds)
# When multiple episodes are downloaded at once, process them one by one with this delay
# Set to 0 to disable sequential processing (parallel mode)
SEQUENTIAL_DELAY=20.0
# API timeout in seconds
TIMEOUT_SECONDS=45
+207
View File
@@ -0,0 +1,207 @@
# Maintainarr Webhook Integration
This document describes how to configure NFOGuard to receive webhooks from Maintainarr for automatic database cleanup when media is removed.
## Overview
When Maintainarr removes media from your Plex/Radarr/Sonarr collections, NFOGuard can automatically remove the corresponding entries from its database to keep it clean and up-to-date.
## Webhook Configuration
### 1. NFOGuard Endpoint
The webhook endpoint is available at:
```
http://YOUR_NFOGUARD_HOST:8080/webhook/maintainarr
```
**Important**: Use the core NFOGuard container port (typically 8080), not the web interface port (8081).
### 2. Maintainarr Configuration
In Maintainarr, create a new webhook notification agent with these settings:
#### Basic Settings
- **Name**: `NFOGuard Cleanup`
- **Enabled**: ✅ Checked
- **Agent**: `Webhook`
#### Webhook Configuration
- **Webhook URL**: `http://YOUR_NFOGUARD_HOST:8080/webhook/maintainarr`
- **JSON Payload**: Use the template below
- **Auth Header**: *(Leave empty - no authentication required)*
#### JSON Payload Template
Copy and paste this JSON template into the "Json Payload" field:
```json
{
"notification_type": "{{notification_type}}",
"subject": "{{subject}}",
"message": "{{message}}",
"extra": "{{extra}}"
}
```
**Important Note**: Maintainarr's template variables may not include IMDb IDs directly. The webhook will attempt to extract IMDb IDs from the notification content, but this may require manual configuration or rule setup in Maintainarr to include IMDb IDs in the notification text.
**Alternative Approach**: If Maintainarr doesn't provide IMDb IDs in notifications, you may need to use NFOGuard's manual cleanup tools or configure Maintainarr rules to include IMDb information in the message content.
#### Event Types
Select these notification types:
-**Media Removed From Collection** - Removes media from NFOGuard database
-**Media About To Be Handled** - Optional: Log when media is about to be processed
Optional event types (will be logged but not processed):
- ☐ Media Added To Collection
- ☐ Media Handled
- ☐ Rule Handling Failed
- ☐ Collection Handling Failed
## Supported Operations
### Movies
When a movie is removed from Maintainarr:
- NFOGuard checks if the movie exists in its database (by IMDb ID)
- If found, removes the movie record from the `movies` table
- Logs the deletion operation
### TV Series
When a TV series is removed from Maintainarr:
- NFOGuard checks if the series exists in its database (by IMDb ID)
- If found, removes all episode records from the `episodes` table
- Removes the series record from the `series` table
- Logs the deletion operation with episode count
## Webhook Payload
Maintainarr sends webhook payloads using template variables that you configure:
```json
{
"notification_type": "Media Removed",
"subject": "Example Movie (2023)",
"message": "Removed movie Example Movie from collection Action Movies - IMDb: tt1234567",
"extra": "tt1234567"
}
```
### How It Works
1. **Maintainarr** populates the template variables ({{notification_type}}, {{subject}}, {{message}}, {{extra}})
2. **NFOGuard** receives the webhook and parses the content to extract:
- **IMDb ID**: Extracted from `message`, `subject`, or `extra` fields using pattern matching
- **Media Type**: Determined from message content keywords or database lookup
- **Title**: Extracted from `subject` or `message` fields
### Media Identification
NFOGuard looks for IMDb IDs in this format:
- `tt1234567` (preferred)
- `1234567` (will be converted to tt1234567)
The webhook handler uses intelligent parsing to:
- Extract IMDb IDs from any field using regex patterns
- Determine if media is a Movie or Series based on keywords or database lookup
- Extract the media title from subject or message content
## Response Format
NFOGuard responds with JSON indicating the result:
### Success Response
```json
{
"status": "success",
"message": "Processed Media Removed for Example Movie",
"media_type": "Movie",
"imdb_id": "tt1234567",
"removed_count": 1,
"removed_items": ["Movie: Example Movie (tt1234567)"]
}
```
### Ignored Response
```json
{
"status": "ignored",
"reason": "Media tt1234567 not found in database"
}
```
### Error Response
```json
{
"status": "error",
"message": "No IMDb ID found in webhook payload"
}
```
## Logging
All webhook activities are logged with details:
```
INFO: Received Maintainarr webhook: Media Removed
INFO: Processing movie deletion for Example Movie (tt1234567)
SUCCESS: Removed movie Example Movie (tt1234567) from database
INFO: Maintainarr cleanup: Media Removed - Movie 'Example Movie' (tt1234567). Removed from database: Movie: Example Movie (tt1234567)
```
## Troubleshooting
### Common Issues
1. **No IMDb ID Found**:
- Maintainarr template variables may not include IMDb IDs
- Check if the notification message contains IMDb information
- You may need to manually include IMDb IDs in Maintainarr rule configurations
2. **Media Not Found**:
- Check if the media exists in NFOGuard's database
- Verify the IMDb ID matches between Maintainarr and NFOGuard
3. **Connection Issues**:
- Ensure NFOGuard core container is accessible on port 8080
- Check firewall settings and network connectivity
4. **Authentication Errors**:
- No authentication is required for the webhook endpoint
- Ensure you're using the core container port, not web interface port
5. **Test Notifications**:
- Test notifications (like the one you just sent) will be acknowledged but not processed
- Real media removal events will trigger the cleanup process
### Testing the Webhook
You can test the webhook manually using curl:
```bash
curl -X POST http://YOUR_NFOGUARD_HOST:8080/webhook/maintainarr \
-H "Content-Type: application/json" \
-d '{
"notification_type": "Media Removed",
"subject": "Test Movie (2023)",
"message": "Removed movie Test Movie from collection - IMDb: tt1234567",
"extra": "tt1234567"
}'
```
## Security Considerations
- The webhook endpoint does not require authentication
- Consider using firewalls or network restrictions to limit access
- The endpoint only processes deletion requests, not additions
- All operations are logged for audit purposes
## Integration Benefits
- **Automatic Cleanup**: Keeps NFOGuard database synchronized with your media collection
- **Accurate Statistics**: Dashboard stats reflect only currently available media
- **Reduced Manual Maintenance**: No need to manually clean up orphaned entries
- **Audit Trail**: All deletions are logged with full details
## Version Compatibility
- NFOGuard: v2.8.0+
- Maintainarr: All versions with webhook support
- Requires NFOGuard core container (processing container), not web-only container
+12
View File
@@ -31,6 +31,18 @@ class RadarrWebhook(BaseModel):
extra = "allow"
class MaintainarrWebhook(BaseModel):
"""Maintainarr webhook payload model - uses template variables"""
notification_type: Optional[str] = None # e.g., "Media Removed"
subject: Optional[str] = None
message: Optional[str] = None
image: Optional[str] = None
extra: Optional[str] = None
class Config:
extra = "allow"
class HealthResponse(BaseModel):
"""Health check response model"""
status: str
+947 -3
View File
@@ -12,7 +12,7 @@ from typing import Optional
# Import models
from api.models import (
SonarrWebhook, RadarrWebhook, HealthResponse, TVSeasonRequest, TVEpisodeRequest,
SonarrWebhook, RadarrWebhook, MaintainarrWebhook, HealthResponse, TVSeasonRequest, TVEpisodeRequest,
MovieUpdateRequest, EpisodeUpdateRequest, BulkUpdateRequest
)
# Web routes removed - handled by separate web container
@@ -107,9 +107,26 @@ async def sonarr_webhook(request: Request, background_tasks: BackgroundTasks, de
if not episodes_data and webhook.episodeFile:
episode_file = webhook.episodeFile
# Extract season and episode from episodeFile if available
# Extract season and episode from episodeFile path/filename
season_num = episode_file.get("seasonNumber")
episode_num = episode_file.get("episodeNumber")
episode_num = episode_file.get("episodeNumber")
# If not directly available, parse from relativePath or path
if not (season_num and episode_num):
from utils.nfo_patterns import extract_episode_info_from_filename
# Try relativePath first, then path
file_path = episode_file.get("relativePath") or episode_file.get("path", "")
print(f"DEBUG: Parsing episode info from path: {file_path}")
episode_info = extract_episode_info_from_filename(file_path)
if episode_info:
season_num = episode_info["season"]
episode_num = episode_info["episode"]
print(f"DEBUG: Extracted from filename - Season: {season_num}, Episode: {episode_num}")
else:
print(f"DEBUG: Could not extract season/episode from filename: {file_path}")
print(f"DEBUG: episodeFile seasonNumber: {season_num}, episodeNumber: {episode_num}")
if season_num and episode_num:
# Create episode data structure that matches what process_webhook_episodes expects
@@ -118,6 +135,7 @@ async def sonarr_webhook(request: Request, background_tasks: BackgroundTasks, de
"episodeNumber": episode_num,
"id": episode_file.get("id"),
"title": episode_file.get("title")
# Note: Not including dateAdded - we use database-first approach with Sonarr fallback
}]
print(f"INFO: Extracted episode info from episodeFile for {webhook.eventType}: S{season_num:02d}E{episode_num:02d}")
else:
@@ -310,6 +328,181 @@ async def radarr_webhook(request: Request, background_tasks: BackgroundTasks, de
return {"status": "error", "message": str(e)}
async def maintainarr_webhook(request: Request, background_tasks: BackgroundTasks, dependencies: dict):
"""Handle Maintainarr webhooks for media deletion"""
db = dependencies["db"]
config = dependencies["config"]
try:
payload = await _read_payload(request)
if not payload:
raise HTTPException(status_code=422, detail="Empty Maintainarr payload")
webhook = MaintainarrWebhook(**payload)
print(f"INFO: Received Maintainarr webhook: {webhook.notification_type}")
print(f"DEBUG: Full Maintainarr webhook payload: {payload}")
# Handle test notifications differently for debugging
notification_type = webhook.notification_type or ""
if notification_type == "TEST_NOTIFICATION":
return {
"status": "test_received",
"message": "Test notification received successfully",
"available_fields": {
"notification_type": webhook.notification_type,
"subject": webhook.subject,
"message": webhook.message,
"extra": webhook.extra
},
"debug": "This is a test notification. Real media removal events will be processed when they occur."
}
# Only process media removal notifications
if "removed" not in notification_type.lower() and "delete" not in notification_type.lower():
return {"status": "ignored", "reason": f"Notification type '{notification_type}' not processed"}
# Parse message to extract media information
message = webhook.message or ""
subject = webhook.subject or ""
extra = webhook.extra or ""
# Try to extract IMDb ID from message, subject, or extra data
import re
imdb_pattern = r'tt\d{7,8}|\b\d{7,8}\b'
imdb_id = None
title = "Unknown Media"
media_type = "Unknown"
# Look for IMDb ID in all fields
for field in [message, subject, extra]:
if field:
imdb_matches = re.findall(imdb_pattern, field)
if imdb_matches:
imdb_id = imdb_matches[0]
if not imdb_id.startswith("tt"):
imdb_id = f"tt{imdb_id}"
break
if not imdb_id:
print(f"WARNING: No IMDb ID found in Maintainarr webhook - Message: '{message}', Subject: '{subject}'")
return {"status": "ignored", "reason": "No IMDb ID found in webhook payload"}
# Try to extract title from subject or message
if subject and subject.strip():
title = subject.strip()
elif message and message.strip():
# Extract title from message if possible
title_match = re.search(r'(?:movie|series|show)\s*[:\-]?\s*(.+?)(?:\s*\(|$)', message, re.IGNORECASE)
if title_match:
title = title_match.group(1).strip()
else:
title = message.strip()
# Try to determine media type from message content
if any(word in message.lower() for word in ["series", "show", "tv", "season", "episode"]):
media_type = "Series"
elif any(word in message.lower() for word in ["movie", "film"]):
media_type = "Movie"
else:
# Try both types - first check if it's a series, then movie
series = db.get_series_by_imdb(imdb_id)
movie = db.get_movie_by_imdb(imdb_id)
if series:
media_type = "Series"
elif movie:
media_type = "Movie"
else:
print(f"INFO: Media {title} ({imdb_id}) not found in database")
return {"status": "ignored", "reason": f"Media {imdb_id} not found in database"}
# Process deletion based on media type
removed_count = 0
removed_items = []
if media_type == "Movie":
print(f"INFO: Processing movie deletion for {title} ({imdb_id})")
# Check if movie exists in database
movie = db.get_movie_by_imdb(imdb_id)
if movie:
if db.delete_movie(imdb_id):
removed_count += 1
removed_items.append(f"Movie: {title} ({imdb_id})")
print(f"SUCCESS: Removed movie {title} ({imdb_id}) from database")
else:
print(f"WARNING: Failed to remove movie {title} ({imdb_id}) from database")
else:
print(f"INFO: Movie {title} ({imdb_id}) not found in database")
elif media_type == "Series":
print(f"INFO: Processing series deletion for {title} ({imdb_id})")
# Check if series exists in database
series = db.get_series_by_imdb(imdb_id)
if series:
# Delete all episodes for the series
episodes_removed = db.delete_series_episodes(imdb_id)
if episodes_removed > 0:
removed_count += episodes_removed
removed_items.append(f"Series episodes: {title} ({imdb_id}) - {episodes_removed} episodes")
print(f"SUCCESS: Removed {episodes_removed} episodes for series {title} ({imdb_id})")
# Delete the series record
if db.delete_series(imdb_id):
removed_count += 1
removed_items.append(f"Series: {title} ({imdb_id})")
print(f"SUCCESS: Removed series {title} ({imdb_id}) from database")
else:
print(f"WARNING: Failed to remove series {title} ({imdb_id}) from database")
else:
print(f"INFO: Series {title} ({imdb_id}) not found in database")
# Log the cleanup operation
if removed_count > 0:
background_tasks.add_task(
_log_maintainarr_cleanup,
notification_type,
media_type,
title,
imdb_id,
removed_items,
webhook.subject
)
return {
"status": "success",
"message": f"Processed {notification_type} for {title}",
"media_type": media_type,
"imdb_id": imdb_id,
"removed_count": removed_count,
"removed_items": removed_items
}
except Exception as e:
print(f"ERROR: Maintainarr webhook error: {e}")
import traceback
print(f"ERROR: Traceback: {traceback.format_exc()}")
return {"status": "error", "message": str(e)}
async def _log_maintainarr_cleanup(event_type: str, media_type: str, title: str, imdb_id: str, removed_items: list, collection_name: str = None):
"""Background task to log Maintainarr cleanup operations"""
try:
log_message = f"Maintainarr cleanup: {event_type} - {media_type} '{title}' ({imdb_id})"
if collection_name:
log_message += f" from collection '{collection_name}'"
log_message += f". Removed from database: {', '.join(removed_items)}"
print(f"INFO: {log_message}")
# Could extend this to write to a cleanup log file or database table
except Exception as e:
print(f"ERROR: Failed to log Maintainarr cleanup: {e}")
async def health(dependencies: dict) -> HealthResponse:
"""Health check endpoint with Radarr database status"""
db = dependencies["db"]
@@ -1890,6 +2083,59 @@ def stop_scan_tracking():
# Route Registration
# ---------------------------
async def update_episode_nfo(imdb_id: str, season: int, episode: int, request: Request, dependencies: dict):
"""Update NFO file for a specific episode with new dateadded"""
try:
# Get request data
payload = await request.json()
dateadded = payload.get('dateadded')
source = payload.get('source', 'manual')
aired = payload.get('aired')
# Get dependencies
config = dependencies["config"]
nfo_manager = dependencies["nfo_manager"]
if not config.manage_nfo:
return {"success": False, "message": "NFO management is disabled"}
# Find the series directory based on IMDb ID
series_path = None
for tv_path in config.tv_paths:
for series_dir in Path(tv_path).iterdir():
if series_dir.is_dir() and imdb_id.lower() in series_dir.name.lower():
series_path = series_dir
break
if series_path:
break
if not series_path:
return {"success": False, "message": f"Series directory not found for {imdb_id}"}
# Get season directory
season_dir = series_path / config.tv_season_dir_format.format(season=season)
if not season_dir.exists():
return {"success": False, "message": f"Season directory not found: {season_dir}"}
# Update NFO file
nfo_manager.create_episode_nfo(
season_dir=season_dir,
season_num=season,
episode_num=episode,
aired=aired,
dateadded=dateadded,
source=source,
lock_metadata=config.lock_metadata
)
print(f"✅ Updated NFO file for {imdb_id} S{season:02d}E{episode:02d}")
return {"success": True, "message": f"NFO file updated for {imdb_id} S{season:02d}E{episode:02d}"}
except Exception as e:
print(f"❌ Error updating NFO file for {imdb_id} S{season:02d}E{episode:02d}: {e}")
return {"success": False, "message": f"Failed to update NFO file: {str(e)}"}
def register_routes(app, dependencies: dict):
"""
Register all routes with the FastAPI app
@@ -1916,6 +2162,10 @@ def register_routes(app, dependencies: dict):
async def _radarr_webhook(request: Request, background_tasks: BackgroundTasks):
return await radarr_webhook(request, background_tasks, dependencies)
@app.post("/webhook/maintainarr")
async def _maintainarr_webhook(request: Request, background_tasks: BackgroundTasks):
return await maintainarr_webhook(request, background_tasks, dependencies)
@app.get("/health")
async def _health() -> HealthResponse:
return await health(dependencies)
@@ -1939,6 +2189,10 @@ def register_routes(app, dependencies: dict):
@app.delete("/database/episode/{imdb_id}/{season}/{episode}")
async def _delete_episode(imdb_id: str, season: int, episode: int):
return await delete_episode(imdb_id, season, episode, dependencies)
@app.post("/api/episodes/{imdb_id}/{season}/{episode}/update-nfo")
async def _update_episode_nfo(imdb_id: str, season: int, episode: int, request: Request):
return await update_episode_nfo(imdb_id, season, episode, request, dependencies)
@app.delete("/database/series/{imdb_id}/episodes")
async def _delete_series_episodes(imdb_id: str):
@@ -2023,6 +2277,696 @@ def register_routes(app, dependencies: dict):
# - Health checks (/health)
#
# Web interface available on separate container port 8081
async def nfo_repair_scan():
"""Scan filesystem for episodes/movies missing dateadded elements in NFO files"""
import os
import xml.etree.ElementTree as ET
config = dependencies.get("config")
db = dependencies.get("db")
if not config or not db:
raise HTTPException(status_code=500, detail=f"Dependencies not available - config:{config is not None}, db:{db is not None}")
print("🔧 Starting NFO repair scan from core container")
import time
scan_start_time = time.time()
missing_items = {
"episodes": [],
"movies": []
}
# Track total files processed for statistics
total_tv_files_checked = 0
total_movie_files_checked = 0
try:
# Use filesystem-first approach: find all NFO files missing dateadded
print("📺 Scanning NFO files directly for missing dateadded elements")
# Get TV and Movie library paths from config
tv_path = getattr(config, 'tv_library_path', '/media/TV')
movie_path = getattr(config, 'movie_library_path', '/media/Movies')
print(f"📁 Scanning TV path: {tv_path}")
print(f"📁 Scanning Movie path: {movie_path}")
import subprocess
import re
# Separate tracking for TV and movie missing files
missing_tv_nfo_files = []
missing_movie_nfo_files = []
# Find all NFO files in TV directory
if os.path.exists(tv_path):
print("🔍 Finding all TV NFO files...")
try:
# Use find to get all NFO files
find_result = subprocess.run(
['find', tv_path, '-name', '*.nfo', '-type', 'f'],
capture_output=True, text=True, timeout=60
)
if find_result.returncode == 0:
tv_nfo_files = find_result.stdout.strip().split('\n')
tv_nfo_files = [f for f in tv_nfo_files if f] # Remove empty strings
print(f"📊 Found {len(tv_nfo_files)} TV NFO files")
# Check each NFO file for missing dateadded
for i, nfo_file in enumerate(tv_nfo_files):
total_tv_files_checked = i + 1 # Track progress
if i % 1000 == 0 and i > 0:
elapsed = time.time() - scan_start_time
print(f"📊 Progress: {i}/{len(tv_nfo_files)} TV NFO files checked in {elapsed:.1f}s, {len(missing_tv_nfo_files)} missing found")
# Timeout check
if time.time() - scan_start_time > 180: # 3 minutes
print(f"⏰ TV NFO scan timeout after 3 minutes - checked {i}/{len(tv_nfo_files)} files")
break
try:
# Use grep to quickly check if dateadded exists
grep_result = subprocess.run(
['grep', '-l', '<dateadded>', nfo_file],
capture_output=True, text=True, timeout=5
)
# If grep returns non-zero, dateadded is missing
if grep_result.returncode != 0:
missing_tv_nfo_files.append(nfo_file)
# Log specific files we're looking for
if 'Hudson' in nfo_file and 'S08E05' in nfo_file:
print(f"🔍 FOUND MISSING: Hudson & Rex S08E05 at {nfo_file}")
if 'Star Trek' in nfo_file and 'S03E07' in nfo_file:
print(f"🔍 FOUND MISSING: Star Trek SNW S03E07 at {nfo_file}")
except subprocess.TimeoutExpired:
print(f"⏰ Timeout checking {nfo_file}")
continue
except Exception as e:
print(f"⚠️ Error checking {nfo_file}: {e}")
continue
else:
print(f"❌ Error finding TV NFO files: {find_result.stderr}")
except subprocess.TimeoutExpired:
print("⏰ Find command timeout for TV files")
except Exception as e:
print(f"❌ Error scanning TV directory: {e}")
else:
print(f"❌ TV path does not exist: {tv_path}")
print(f"📊 Found {len(missing_tv_nfo_files)} TV NFO files missing dateadded elements")
# Convert TV file paths to episode information with direct NFO parsing
def extract_imdb_from_nfo_content(nfo_file, is_episode=True):
"""Extract IMDb ID directly from NFO file content - we already know the file exists"""
imdb_id = "unknown"
try:
# Try to parse the NFO file we already know exists (since grep found it missing dateadded)
tree = ET.parse(nfo_file)
root = tree.getroot()
# Method 1: Check <imdb> tag
imdb_elem = root.find("imdb")
if imdb_elem is not None and imdb_elem.text:
imdb_text = imdb_elem.text.strip()
if imdb_text.startswith('tt'):
print(f"✅ Found IMDb {imdb_text} in <imdb> tag: {os.path.basename(nfo_file)}")
return imdb_text
# Method 2: Check <uniqueid type="imdb"> tags
for uniqueid in root.findall("uniqueid"):
if uniqueid.get("type") == "imdb" and uniqueid.text:
imdb_text = uniqueid.text.strip()
if imdb_text.startswith('tt'):
print(f"✅ Found IMDb {imdb_text} in <uniqueid type='imdb'>: {os.path.basename(nfo_file)}")
return imdb_text
# Method 3: Check for IMDb pattern anywhere in NFO content
nfo_content = ET.tostring(root, encoding='unicode')
content_match = re.search(r'(tt\d{7,})', nfo_content)
if content_match:
imdb_text = content_match.group(1)
print(f"✅ Found IMDb {imdb_text} in NFO content: {os.path.basename(nfo_file)}")
return imdb_text
except ET.ParseError as e:
# Handle malformed XML by trying text-based search
print(f"⚠️ NFO XML parse error in {os.path.basename(nfo_file)}: {e}")
try:
# Fallback: Read as text and search for IMDb patterns
with open(nfo_file, 'r', encoding='utf-8', errors='ignore') as f:
nfo_text = f.read()
# Search for IMDb patterns in raw text
text_match = re.search(r'(tt\d{7,})', nfo_text)
if text_match:
imdb_text = text_match.group(1)
print(f"✅ Found IMDb {imdb_text} in NFO text content: {os.path.basename(nfo_file)}")
return imdb_text
except Exception as text_error:
print(f"⚠️ Error reading NFO as text {nfo_file}: {text_error}")
except Exception as e:
print(f"⚠️ Unexpected error parsing NFO {nfo_file}: {e}")
# Method 4: Fallback - check folder structure
if is_episode:
series_folder = os.path.dirname(os.path.dirname(nfo_file))
folder_name = os.path.basename(series_folder)
folder_match = re.search(r'\[imdb-(tt\d+)\]', folder_name, re.IGNORECASE)
if folder_match:
print(f"✅ Found IMDb {folder_match.group(1)} in folder: {folder_name}")
return folder_match.group(1)
else:
# For movies, check movie folder
movie_folder = os.path.dirname(nfo_file)
folder_name = os.path.basename(movie_folder)
folder_match = re.search(r'\[imdb-(tt\d+)\]', folder_name, re.IGNORECASE)
if folder_match:
print(f"✅ Found IMDb {folder_match.group(1)} in folder: {folder_name}")
return folder_match.group(1)
print(f"❌ No IMDb ID found in {os.path.basename(nfo_file)}")
return "unknown"
for nfo_file in missing_tv_nfo_files:
try:
# Parse episode info from file path
# Example: /media/TV/tv/Hudson & Rex (2019) [imdb-tt9111220]/Season 08/Hudson & Rex (2019)-S08E05-Episode.nfo
path_parts = nfo_file.split('/')
# Look for season/episode pattern
season_match = re.search(r'Season\s+(\d+)', nfo_file, re.IGNORECASE)
episode_match = re.search(r'S(\d+)E(\d+)', os.path.basename(nfo_file), re.IGNORECASE)
if season_match and episode_match:
season = int(episode_match.group(1))
episode = int(episode_match.group(2))
# Extract series name and path
series_path = os.path.dirname(os.path.dirname(nfo_file))
series_name = os.path.basename(series_path)
# Extract IMDb ID from NFO content
imdb_id = extract_imdb_from_nfo_content(nfo_file, is_episode=True)
# Clean series name (remove year and imdb parts)
clean_series_name = re.sub(r'\s*\(\d{4}\)\s*\[imdb-[^]]+\]', '', series_name).strip()
missing_items["episodes"].append({
"imdb_id": imdb_id,
"season": season,
"episode": episode,
"series_name": clean_series_name,
"series_path": series_path,
"dateadded": None,
"nfo_path": nfo_file,
"reason": "NFO missing dateadded element (filesystem scan)"
})
except Exception as e:
print(f"⚠️ Error parsing NFO file path {nfo_file}: {e}")
continue
# Scan movies using filesystem grep approach
print("🎬 Scanning movie NFO files for missing dateadded elements")
# Find all NFO files in Movie directory
if os.path.exists(movie_path):
print(f"🔍 Finding all Movie NFO files in {movie_path}...")
try:
# Use find to get all NFO files
find_result = subprocess.run(
['find', movie_path, '-name', '*.nfo', '-type', 'f'],
capture_output=True, text=True, timeout=60
)
if find_result.returncode == 0:
movie_nfo_files = find_result.stdout.strip().split('\n')
movie_nfo_files = [f for f in movie_nfo_files if f] # Remove empty strings
print(f"📊 Found {len(movie_nfo_files)} Movie NFO files")
# Check each NFO file for missing dateadded
for i, nfo_file in enumerate(movie_nfo_files):
total_movie_files_checked = i + 1 # Track progress
if i % 500 == 0 and i > 0:
elapsed = time.time() - scan_start_time
print(f"📊 Progress: {i}/{len(movie_nfo_files)} Movie NFO files checked in {elapsed:.1f}s, {len(missing_movie_nfo_files)} missing found")
# Timeout check
if time.time() - scan_start_time > 300: # 5 minutes total
print(f"⏰ Movie NFO scan timeout after 5 minutes - checked {i}/{len(movie_nfo_files)} files")
break
try:
# Use grep to quickly check if dateadded exists
grep_result = subprocess.run(
['grep', '-l', '<dateadded>', nfo_file],
capture_output=True, text=True, timeout=5
)
# If grep returns non-zero, dateadded is missing
if grep_result.returncode != 0:
missing_movie_nfo_files.append(nfo_file)
except subprocess.TimeoutExpired:
print(f"⏰ Timeout checking {nfo_file}")
continue
except Exception as e:
print(f"⚠️ Error checking {nfo_file}: {e}")
continue
else:
print(f"❌ Error finding Movie NFO files: {find_result.stderr}")
except subprocess.TimeoutExpired:
print("⏰ Find command timeout for Movie files")
except Exception as e:
print(f"❌ Error scanning Movie directory: {e}")
else:
print(f"❌ Movie path does not exist: {movie_path}")
# Process missing movie NFO files
print(f"📊 Found {len(missing_movie_nfo_files)} Movie NFO files missing dateadded elements")
for nfo_file in missing_movie_nfo_files:
try:
# Extract movie info from file path
# Example: /media/Movies/movies/Knives Out (2019) [imdb-tt8946378]/movie.nfo
movie_dir = os.path.dirname(nfo_file)
movie_folder_name = os.path.basename(movie_dir)
# Extract IMDb ID from NFO content
imdb_id = extract_imdb_from_nfo_content(nfo_file, is_episode=False)
# Clean movie title (remove year and imdb parts)
clean_movie_title = re.sub(r'\s*\(\d{4}\)\s*\[imdb-[^]]+\]', '', movie_folder_name).strip()
missing_items["movies"].append({
"imdb_id": imdb_id,
"title": clean_movie_title,
"path": movie_dir,
"dateadded": None,
"nfo_path": nfo_file,
"reason": "NFO missing dateadded element (filesystem scan)"
})
except Exception as e:
print(f"⚠️ Error parsing movie NFO file path {nfo_file}: {e}")
continue
# Database lookup to get actual dateadded values for missing items
print("🔍 Looking up dateadded values from database for missing items...")
# Enhance episodes with database data
episodes_with_dates = 0
episodes_missing_db = 0
for episode in missing_items["episodes"]:
if episode["imdb_id"] != "unknown":
try:
with db.get_connection() as conn:
cursor = conn.cursor()
cursor.execute("""
SELECT dateadded, source
FROM episodes
WHERE imdb_id = %s AND season = %s AND episode = %s
LIMIT 1
""", (episode["imdb_id"], episode["season"], episode["episode"]))
result = cursor.fetchone()
if result:
episode["dateadded"] = result["dateadded"]
episode["source"] = result.get("source", "database")
episodes_with_dates += 1
print(f"✅ Found database date for {episode['imdb_id']} S{episode['season']:02d}E{episode['episode']:02d}: {result['dateadded']}")
else:
episodes_missing_db += 1
# Try to find other episodes from same series to estimate date
cursor.execute("""
SELECT dateadded, season, episode
FROM episodes
WHERE imdb_id = %s AND dateadded IS NOT NULL
ORDER BY season DESC, episode DESC
LIMIT 3
""", (episode["imdb_id"],))
similar_episodes = cursor.fetchall()
if similar_episodes:
latest_date = similar_episodes[0]["dateadded"]
episode["dateadded"] = latest_date
episode["source"] = f"estimated_from_series_latest"
print(f"🔮 Estimated date for {episode['imdb_id']} S{episode['season']:02d}E{episode['episode']:02d} from latest episode: {latest_date}")
else:
# Use NFO file modification time as fallback
try:
import os
from datetime import datetime
if os.path.exists(episode["nfo_path"]):
file_mtime = os.path.getmtime(episode["nfo_path"])
file_date = datetime.fromtimestamp(file_mtime)
episode["dateadded"] = file_date
episode["source"] = "nfo_file_mtime"
print(f"📅 Using NFO file date for {episode['imdb_id']} S{episode['season']:02d}E{episode['episode']:02d}: {file_date}")
else:
print(f"❌ No fallback date available for {episode['imdb_id']} S{episode['season']:02d}E{episode['episode']:02d}")
except Exception as fallback_error:
print(f"⚠️ Fallback date lookup failed for {episode['imdb_id']}: {fallback_error}")
except Exception as e:
print(f"⚠️ Database lookup failed for episode {episode['imdb_id']}: {e}")
# Enhance movies with database data
movies_with_dates = 0
movies_missing_db = 0
for movie in missing_items["movies"]:
if movie["imdb_id"] != "unknown":
try:
with db.get_connection() as conn:
cursor = conn.cursor()
cursor.execute("""
SELECT dateadded, source
FROM movies
WHERE imdb_id = %s
LIMIT 1
""", (movie["imdb_id"],))
result = cursor.fetchone()
if result:
movie["dateadded"] = result["dateadded"]
movie["source"] = result.get("source", "database")
movies_with_dates += 1
print(f"✅ Found database date for movie {movie['imdb_id']}: {result['dateadded']}")
else:
movies_missing_db += 1
# Use NFO file modification time as fallback for movies
try:
import os
from datetime import datetime
if os.path.exists(movie["nfo_path"]):
file_mtime = os.path.getmtime(movie["nfo_path"])
file_date = datetime.fromtimestamp(file_mtime)
movie["dateadded"] = file_date
movie["source"] = "nfo_file_mtime"
print(f"📅 Using NFO file date for movie {movie['imdb_id']}: {file_date}")
else:
print(f"❌ No fallback date available for movie {movie['imdb_id']}")
except Exception as fallback_error:
print(f"⚠️ Fallback date lookup failed for movie {movie['imdb_id']}: {fallback_error}")
except Exception as e:
print(f"⚠️ Database lookup failed for movie {movie['imdb_id']}: {e}")
# Print summary statistics
total_episodes = len(missing_items["episodes"])
total_movies = len(missing_items["movies"])
print(f"📊 Database lookup summary:")
print(f" Episodes: {episodes_with_dates}/{total_episodes} found in DB, {episodes_missing_db} missing from DB")
print(f" Movies: {movies_with_dates}/{total_movies} found in DB, {movies_missing_db} missing from DB")
total_missing = len(missing_items["episodes"]) + len(missing_items["movies"])
print(f"✅ NFO repair scan complete: {total_missing} items missing dateadded elements")
return {
"status": "success",
"total_missing": total_missing,
"episodes_missing": len(missing_items["episodes"]),
"movies_missing": len(missing_items["movies"]),
"total_tv_files_checked": total_tv_files_checked,
"total_movie_files_checked": total_movie_files_checked,
"missing_items": missing_items
}
except Exception as e:
print(f"❌ Error during NFO repair scan: {str(e)}")
raise HTTPException(status_code=500, detail=f"NFO repair scan failed: {str(e)}")
@app.get("/admin/nfo-repair-scan")
async def _nfo_repair_scan():
return await nfo_repair_scan()
async def nfo_repair_fix():
"""Fix missing dateadded elements in NFO files using database values"""
import os
import xml.etree.ElementTree as ET
from datetime import datetime
config = dependencies.get("config")
db = dependencies.get("db")
nfo_manager = dependencies.get("nfo_manager")
if not config or not db:
raise HTTPException(status_code=500, detail=f"Dependencies not available")
print("🔧 Starting NFO repair fix process from core container")
# First, run the scan to identify missing items
scan_result = await nfo_repair_scan()
if scan_result["status"] != "success":
return {"status": "error", "message": "Scan failed"}
missing_items = scan_result["missing_items"]
total_episodes = len(missing_items["episodes"])
total_movies = len(missing_items["movies"])
print(f"🔍 Found {total_episodes} episodes and {total_movies} movies to fix")
fixed_count = 0
failed_count = 0
results = []
# Fix episodes
for episode in missing_items["episodes"]:
if episode.get("dateadded") and episode.get("imdb_id") != "unknown":
try:
nfo_path = episode["nfo_path"]
dateadded_value = episode["dateadded"]
# Format dateadded for NFO file
if isinstance(dateadded_value, str):
# Parse string datetime
dt = datetime.fromisoformat(dateadded_value.replace('Z', '+00:00'))
else:
dt = dateadded_value
# Format as NFO expects: 2024-10-15 12:34:56
nfo_dateadded = dt.strftime('%Y-%m-%d %H:%M:%S')
# Read and modify NFO file
if os.path.exists(nfo_path):
tree = ET.parse(nfo_path)
root = tree.getroot()
# Remove existing dateadded element if present
existing = root.find("dateadded")
if existing is not None:
root.remove(existing)
# Add new dateadded element
dateadded_elem = ET.SubElement(root, "dateadded")
dateadded_elem.text = nfo_dateadded
# Add source attribution (similar to what NFOGuard normally adds)
# Check if source element exists, if not create it
source_elem = root.find("source")
if source_elem is None:
source_elem = ET.SubElement(root, "source")
source_elem.text = f"NFOGuard repair - {episode.get('source', 'database')}"
# Add addedby attribution if not present
addedby_elem = root.find("addedby")
if addedby_elem is None:
addedby_elem = ET.SubElement(root, "addedby")
addedby_elem.text = "NFOGuard"
# Write back to file
tree.write(nfo_path, encoding='utf-8', xml_declaration=True)
# VERIFY the write was successful by re-reading the file
try:
verify_tree = ET.parse(nfo_path)
verify_root = verify_tree.getroot()
verify_dateadded = verify_root.find("dateadded")
if verify_dateadded is not None and verify_dateadded.text and verify_dateadded.text.strip():
# Verification successful
fixed_count += 1
results.append({
"type": "episode",
"imdb_id": episode["imdb_id"],
"season": episode["season"],
"episode": episode["episode"],
"nfo_path": nfo_path,
"dateadded": nfo_dateadded,
"verified_dateadded": verify_dateadded.text.strip(),
"status": "fixed_and_verified"
})
print(f"✅ Fixed and verified episode {episode['imdb_id']} S{episode['season']:02d}E{episode['episode']:02d}")
else:
# Verification failed - dateadded missing after write
failed_count += 1
results.append({
"type": "episode",
"imdb_id": episode["imdb_id"],
"season": episode["season"],
"episode": episode["episode"],
"nfo_path": nfo_path,
"dateadded": nfo_dateadded,
"status": "write_failed_verification"
})
print(f"❌ Write verification FAILED for episode {episode['imdb_id']} S{episode['season']:02d}E{episode['episode']:02d} - dateadded not found after write")
except Exception as verify_error:
# Verification failed due to parse error
failed_count += 1
results.append({
"type": "episode",
"imdb_id": episode["imdb_id"],
"season": episode["season"],
"episode": episode["episode"],
"nfo_path": nfo_path,
"status": "verification_parse_error",
"error": str(verify_error)
})
print(f"❌ Verification parse error for episode {episode['imdb_id']} S{episode['season']:02d}E{episode['episode']:02d}: {verify_error}")
else:
failed_count += 1
print(f"❌ NFO file not found: {nfo_path}")
except Exception as e:
failed_count += 1
print(f"❌ Failed to fix episode {episode.get('imdb_id', 'unknown')}: {e}")
# Fix movies
for movie in missing_items["movies"]:
if movie.get("dateadded") and movie.get("imdb_id") != "unknown":
try:
nfo_path = movie["nfo_path"]
dateadded_value = movie["dateadded"]
# Format dateadded for NFO file
if isinstance(dateadded_value, str):
dt = datetime.fromisoformat(dateadded_value.replace('Z', '+00:00'))
else:
dt = dateadded_value
nfo_dateadded = dt.strftime('%Y-%m-%d %H:%M:%S')
# Read and modify NFO file
if os.path.exists(nfo_path):
tree = ET.parse(nfo_path)
root = tree.getroot()
# Remove existing dateadded element if present
existing = root.find("dateadded")
if existing is not None:
root.remove(existing)
# Add new dateadded element
dateadded_elem = ET.SubElement(root, "dateadded")
dateadded_elem.text = nfo_dateadded
# Add source attribution (similar to what NFOGuard normally adds)
source_elem = root.find("source")
if source_elem is None:
source_elem = ET.SubElement(root, "source")
source_elem.text = f"NFOGuard repair - {movie.get('source', 'database')}"
# Add addedby attribution if not present
addedby_elem = root.find("addedby")
if addedby_elem is None:
addedby_elem = ET.SubElement(root, "addedby")
addedby_elem.text = "NFOGuard"
# Write back to file
tree.write(nfo_path, encoding='utf-8', xml_declaration=True)
# VERIFY the write was successful by re-reading the file
try:
verify_tree = ET.parse(nfo_path)
verify_root = verify_tree.getroot()
verify_dateadded = verify_root.find("dateadded")
if verify_dateadded is not None and verify_dateadded.text and verify_dateadded.text.strip():
# Verification successful
fixed_count += 1
results.append({
"type": "movie",
"imdb_id": movie["imdb_id"],
"title": movie["title"],
"nfo_path": nfo_path,
"dateadded": nfo_dateadded,
"verified_dateadded": verify_dateadded.text.strip(),
"status": "fixed_and_verified"
})
print(f"✅ Fixed and verified movie {movie['imdb_id']} ({movie['title']})")
else:
# Verification failed - dateadded missing after write
failed_count += 1
results.append({
"type": "movie",
"imdb_id": movie["imdb_id"],
"title": movie["title"],
"nfo_path": nfo_path,
"dateadded": nfo_dateadded,
"status": "write_failed_verification"
})
print(f"❌ Write verification FAILED for movie {movie['imdb_id']} ({movie['title']}) - dateadded not found after write")
except Exception as verify_error:
# Verification failed due to parse error
failed_count += 1
results.append({
"type": "movie",
"imdb_id": movie["imdb_id"],
"title": movie["title"],
"nfo_path": nfo_path,
"status": "verification_parse_error",
"error": str(verify_error)
})
print(f"❌ Verification parse error for movie {movie['imdb_id']} ({movie['title']}): {verify_error}")
else:
failed_count += 1
print(f"❌ NFO file not found: {nfo_path}")
except Exception as e:
failed_count += 1
print(f"❌ Failed to fix movie {movie.get('imdb_id', 'unknown')}: {e}")
print(f"✅ NFO repair fix complete: {fixed_count} fixed, {failed_count} failed")
# Final summary with detailed statistics
verification_summary = {
"fixed_and_verified": sum(1 for r in results if r.get("status") == "fixed_and_verified"),
"write_failed_verification": sum(1 for r in results if r.get("status") == "write_failed_verification"),
"verification_parse_error": sum(1 for r in results if r.get("status") == "verification_parse_error")
}
print(f"📊 Final summary: {fixed_count} fixed, {failed_count} failed")
print(f" ✅ Verified successful: {verification_summary['fixed_and_verified']}")
print(f" ❌ Write verification failed: {verification_summary['write_failed_verification']}")
print(f" ⚠️ Verification parse errors: {verification_summary['verification_parse_error']}")
return {
"status": "success",
"total_processed": total_episodes + total_movies,
"fixed_count": fixed_count,
"failed_count": failed_count,
"verification_summary": verification_summary,
"results": results[:20] # Show first 20 for UI
}
@app.post("/admin/nfo-repair-fix")
async def _nfo_repair_fix():
return await nfo_repair_fix()
# ---------------------------
# Core API - No Web Interface
# ---------------------------
+479 -49
View File
@@ -93,11 +93,11 @@ async def get_movies_list(dependencies: dict,
params.append(source_filter)
if search:
where_conditions.append("(imdb_id LIKE %s OR path LIKE %s)")
where_conditions.append("(imdb_id ILIKE %s OR path ILIKE %s)")
params.extend([f"%{search}%", f"%{search}%"])
if imdb_search:
where_conditions.append("imdb_id LIKE %s")
where_conditions.append("imdb_id ILIKE %s")
params.append(f"%{imdb_search}%")
where_clause = " AND ".join(where_conditions) if where_conditions else "1=1"
@@ -162,11 +162,11 @@ async def get_tv_series_list(dependencies: dict,
having_conditions = []
if search:
where_conditions.append("(s.imdb_id LIKE %s OR s.path LIKE %s)")
where_conditions.append("(s.imdb_id ILIKE %s OR s.path ILIKE %s)")
params.extend([f"%{search}%", f"%{search}%"])
if imdb_search:
where_conditions.append("s.imdb_id LIKE %s")
where_conditions.append("s.imdb_id ILIKE %s")
params.append(f"%{imdb_search}%")
if source_filter:
@@ -581,11 +581,25 @@ async def update_episode_date(dependencies: dict, imdb_id: str, season: int, epi
"""Update dateadded for a specific episode"""
db = dependencies["db"]
print(f"🔍 DEBUG: update_episode_date called with dateadded={repr(dateadded)}, source={repr(source)}")
# Get existing episode
episode_data = db.get_episode_date(imdb_id, season, episode)
if not episode_data:
raise HTTPException(status_code=404, detail="Episode not found")
# Handle special sources
if source == "airdate" and episode_data.get('aired'):
# Use aired date as dateadded for airdate source
aired_date = episode_data.get('aired')
if hasattr(aired_date, 'isoformat'):
dateadded = aired_date.isoformat() + "T20:00:00" # Set to 8 PM on air date
else:
dateadded = str(aired_date) + "T20:00:00"
print(f"🔍 DEBUG: Using air date as dateadded: {dateadded}")
print(f"🔍 DEBUG: Final dateadded value: {repr(dateadded)}")
# Update the date
db.upsert_episode_date(
imdb_id=imdb_id,
@@ -597,42 +611,46 @@ async def update_episode_date(dependencies: dict, imdb_id: str, season: int, epi
has_video_file=episode_data.get('has_video_file', False)
)
# Create/update NFO file with new data
nfo_manager = dependencies["nfo_manager"]
config = dependencies["config"]
if config.manage_nfo:
try:
# Find the series directory based on IMDb ID
series_path = None
for tv_path in config.tv_paths:
for series_dir in Path(tv_path).iterdir():
if series_dir.is_dir() and imdb_id.lower() in series_dir.name.lower():
series_path = series_dir
break
if series_path:
break
if series_path:
season_dir = series_path / config.tv_season_dir_format.format(season=season)
if season_dir.exists():
nfo_manager.create_episode_nfo(
season_dir=season_dir,
season_num=season,
episode_num=episode,
aired=episode_data.get('aired'),
dateadded=dateadded,
source=source,
lock_metadata=config.lock_metadata
)
print(f"✅ Updated NFO file for {imdb_id} S{season:02d}E{episode:02d}")
else:
print(f"⚠️ Season directory not found: {season_dir}")
# Trigger NFO file update via core container
try:
import urllib.request
import urllib.parse
import json
import os
# Get core container connection details
core_host = os.environ.get("CORE_API_HOST", "nfoguard")
core_port = os.environ.get("CORE_API_PORT", "8080")
# Call core container to update NFO file
nfo_update_url = f"http://{core_host}:{core_port}/api/episodes/{imdb_id}/{season}/{episode}/update-nfo"
# Create request data
request_data = {
"dateadded": dateadded,
"source": source,
"aired": episode_data.get('aired').isoformat() if episode_data.get('aired') else None
}
# Make POST request to core container
req = urllib.request.Request(
nfo_update_url,
data=json.dumps(request_data).encode('utf-8'),
headers={'Content-Type': 'application/json'},
method='POST'
)
with urllib.request.urlopen(req, timeout=10) as response:
if response.status == 200:
print(f"✅ Database and NFO file updated for {imdb_id} S{season:02d}E{episode:02d}")
else:
print(f"⚠️ Series directory not found for {imdb_id}")
print(f"✅ Database updated for {imdb_id} S{season:02d}E{episode:02d}")
print(f"⚠️ NFO file update failed (HTTP {response.status})")
except Exception as e:
print(f"❌ Error updating NFO file for {imdb_id} S{season:02d}E{episode:02d}: {e}")
except Exception as e:
print(f"✅ Database updated for {imdb_id} S{season:02d}E{episode:02d}")
print(f"⚠️ NFO file update failed: {e}")
print(f"️ NFO will be updated on next scan")
# Add to processing history
db.add_processing_history(
@@ -1237,15 +1255,40 @@ def register_web_routes(app, dependencies):
@app.post("/api/movies/{imdb_id}/update-date")
async def api_update_movie_date(imdb_id: str, dateadded: str = None, source: str = "manual"):
return {"error": "Updates not available in web container. Use core container on port 8085."}
return await update_movie_date(dependencies, imdb_id, dateadded, source)
@app.put("/api/movies/{imdb_id}")
async def api_update_movie(imdb_id: str, dateadded: str = None, source: str = "manual"):
return {"error": "Updates not available in web container. Use core container on port 8085."}
return await update_movie_date(dependencies, imdb_id, dateadded, source)
@app.get("/api/movies/{imdb_id}/date-options")
async def api_movie_date_options(imdb_id: str):
return {"options": [], "message": "Date options not available in web container. Use core container on port 8085."}
return await get_movie_date_options(dependencies, imdb_id)
@app.delete("/api/movies/{imdb_id}")
async def api_delete_movie(imdb_id: str):
"""Delete a movie from the database"""
db = dependencies["db"]
try:
# Use the existing database method
deleted = db.delete_movie(imdb_id)
if deleted:
return {
"success": True,
"status": "success",
"message": f"Deleted movie {imdb_id}",
"imdb_id": imdb_id
}
else:
raise HTTPException(status_code=404, detail="Movie not found")
except HTTPException:
raise # Re-raise HTTP exceptions
except Exception as e:
print(f"❌ Error deleting movie: {e}")
raise HTTPException(status_code=500, detail=f"Failed to delete movie: {str(e)}")
# TV series endpoints
@app.get("/api/series")
@@ -1269,16 +1312,50 @@ def register_web_routes(app, dependencies):
@app.post("/api/episodes/{imdb_id}/{season}/{episode}/update-date")
async def api_update_episode_date(imdb_id: str, season: int, episode: int,
dateadded: str = None, source: str = "manual"):
return {"error": "Updates not available in web container. Use core container on port 8085."}
return await update_episode_date(dependencies, imdb_id, season, episode, dateadded, source)
@app.put("/api/episodes/{imdb_id}/{season}/{episode}")
async def api_update_episode(imdb_id: str, season: int, episode: int,
dateadded: str = None, source: str = "manual"):
return {"error": "Updates not available in web container. Use core container on port 8085."}
async def api_update_episode(imdb_id: str, season: int, episode: int, request: Request):
try:
# Parse JSON body
data = await request.json()
dateadded = data.get('dateadded')
source = data.get('source', 'manual')
return await update_episode_date(dependencies, imdb_id, season, episode, dateadded, source)
except Exception as e:
print(f"❌ Error parsing episode update request: {e}")
raise HTTPException(status_code=400, detail=f"Invalid request format: {str(e)}")
@app.get("/api/episodes/{imdb_id}/{season}/{episode}/date-options")
async def api_episode_date_options(imdb_id: str, season: int, episode: int):
return {"options": [], "message": "Date options not available in web container. Use core container on port 8085."}
return await get_episode_date_options(dependencies, imdb_id, season, episode)
@app.delete("/api/episodes/{imdb_id}/{season}/{episode}")
async def api_delete_episode(imdb_id: str, season: int, episode: int):
"""Delete an episode from the database"""
db = dependencies["db"]
try:
# Use the existing database method
deleted = db.delete_episode(imdb_id, season, episode)
if deleted:
return {
"success": True,
"status": "success",
"message": f"Deleted episode {imdb_id} S{season:02d}E{episode:02d}",
"imdb_id": imdb_id,
"season": season,
"episode": episode
}
else:
raise HTTPException(status_code=404, detail="Episode not found")
except HTTPException:
raise # Re-raise HTTP exceptions
except Exception as e:
print(f"❌ Error deleting episode: {e}")
raise HTTPException(status_code=500, detail=f"Failed to delete episode: {str(e)}")
# Bulk operations
@app.post("/api/bulk/update-source")
@@ -1335,7 +1412,7 @@ def register_web_routes(app, dependencies):
import socket
# Get core container URL
core_host = os.environ.get("CORE_API_HOST", "nfoguard-core")
core_host = os.environ.get("CORE_API_HOST", "nfoguard")
core_port = os.environ.get("CORE_API_PORT", "8080")
# Forward query parameters from the request
@@ -1383,7 +1460,7 @@ def register_web_routes(app, dependencies):
import socket
# Get core container connection details
core_host = os.environ.get("CORE_API_HOST", "nfoguard-core")
core_host = os.environ.get("CORE_API_HOST", "nfoguard")
core_port = os.environ.get("CORE_API_PORT", "8080")
try:
@@ -1406,4 +1483,357 @@ def register_web_routes(app, dependencies):
except json.JSONDecodeError:
return {"scanning": False, "message": "Invalid response from core container"}
except Exception as e:
return {"scanning": False, "message": f"Unable to check scan status: {str(e)}"}
return {"scanning": False, "message": f"Unable to check scan status: {str(e)}"}
# Register database admin routes
register_database_admin_routes(app, dependencies)
# =============================================================================
# DATABASE ADMIN INTERFACE
# =============================================================================
async def execute_database_query(dependencies: dict, query: str, limit: int = 100):
"""Execute a database query and return results"""
db = dependencies["db"]
try:
with db.get_connection() as conn:
cursor = conn.cursor()
# Add LIMIT if not already present in query
if "LIMIT" not in query.upper() and not query.strip().endswith(";"):
query += f" LIMIT {limit}"
elif query.strip().endswith(";"):
query = query.strip()[:-1] + f" LIMIT {limit};"
cursor.execute(query)
if query.strip().upper().startswith("SELECT"):
results = cursor.fetchall()
columns = [desc[0] for desc in cursor.description] if cursor.description else []
return {
"success": True,
"query": query,
"columns": columns,
"rows": [dict(zip(columns, row)) for row in results],
"row_count": len(results)
}
else:
conn.commit()
return {
"success": True,
"query": query,
"message": f"Query executed successfully. Rows affected: {cursor.rowcount}",
"rows_affected": cursor.rowcount
}
except Exception as e:
return {
"success": False,
"query": query,
"error": str(e),
"message": f"Database query failed: {str(e)}"
}
async def get_episodes_missing_nfo_dateadded(dependencies: dict):
"""Find episodes and movies missing dateadded elements in NFO files via core container"""
config = dependencies["config"]
try:
print("🔧 Calling core container for NFO repair scan...")
# Call the core container's NFO repair scan endpoint
import aiohttp
import asyncio
core_url = f"http://nfoguard:{config.core_api_port}/admin/nfo-repair-scan"
print(f"🔍 DEBUG: Calling core container at: {core_url}")
timeout = aiohttp.ClientTimeout(total=300.0) # 5 minute timeout for filesystem scan
async with aiohttp.ClientSession(timeout=timeout) as session:
async with session.get(core_url) as response:
if response.status == 200:
result = await response.json()
print(f"✅ Core container scan complete: {result['total_missing']} items missing dateadded")
# Transform the data to match the expected legacy format
episodes_missing = result['missing_items']['episodes']
movies_missing = result['missing_items']['movies']
# Combine for legacy items format
all_missing = []
for episode in episodes_missing:
all_missing.append({
"imdb_id": episode["imdb_id"],
"season": episode["season"],
"episode": episode["episode"],
"series_name": episode["series_name"],
"series_path": episode.get("series_path"),
"dateadded": episode["dateadded"],
"nfo_path": episode["nfo_path"],
"reason": episode.get("reason", "NFO missing dateadded element"),
"media_type": "episode",
"nfo_exists": True,
"dateadded_in_nfo": False,
"should_have_date": True
})
for movie in movies_missing:
all_missing.append({
"imdb_id": movie["imdb_id"],
"title": movie["title"],
"dateadded": movie["dateadded"],
"nfo_path": movie["nfo_path"],
"media_type": "movie",
"nfo_exists": True,
"dateadded_in_nfo": False,
"should_have_date": True
})
return {
"success": True,
"total_episodes_checked": result.get('total_tv_files_checked', 0),
"total_movies_checked": result.get('total_movie_files_checked', 0),
"total_items_checked": result.get('total_tv_files_checked', 0) + result.get('total_movie_files_checked', 0),
"missing_dateadded_count": result['total_missing'],
"items": all_missing[:50], # Limit display to first 50
"debug_info": {
"scan_method": "core_container_filesystem",
"core_container_response": "success",
"episodes_missing": result['episodes_missing'],
"movies_missing": result['movies_missing'],
"core_response_keys": list(result.keys())
}
}
else:
error_text = await response.text()
print(f"❌ Core container scan failed: {response.status} - {error_text}")
return {
"success": False,
"error": f"Core container scan failed: {response.status}",
"message": f"Failed to check NFO files via core container: {response.status}"
}
except Exception as e:
print(f"❌ Error calling core container for NFO repair scan: {str(e)}")
import traceback
traceback.print_exc()
return {
"success": False,
"error": f"Core container communication failed: {str(e)}",
"message": f"Failed to check NFO files: {str(e)}"
}
async def fix_nfo_missing_dateadded(dependencies: dict):
"""Fix NFO files missing dateadded elements via core container"""
config = dependencies["config"]
try:
print("🔧 Calling core container for NFO repair fix...")
# Call the core container's NFO repair fix endpoint
import aiohttp
import asyncio
core_url = f"http://nfoguard:{config.core_api_port}/admin/nfo-repair-fix"
print(f"🔍 DEBUG: Calling core container at: {core_url}")
timeout = aiohttp.ClientTimeout(total=600.0) # 10 minute timeout for fix operation
async with aiohttp.ClientSession(timeout=timeout) as session:
async with session.post(core_url) as response:
if response.status == 200:
result = await response.json()
print(f"✅ Core container fix complete: {result.get('fixed_count', 0)} fixed, {result.get('failed_count', 0)} failed")
return {
"success": True,
"fixed_count": result.get("fixed_count", 0),
"failed_count": result.get("failed_count", 0),
"total_processed": result.get("total_processed", 0),
"results": result.get("results", []),
"message": f"Successfully fixed {result.get('fixed_count', 0)} NFO files"
}
else:
error_text = await response.text()
print(f"❌ Core container fix failed: {response.status} - {error_text}")
return {
"success": False,
"error": f"Core container fix failed: {response.status}",
"message": f"Failed to fix NFO files: {error_text}"
}
except Exception as e:
print(f"❌ Error calling core container for NFO repair fix: {str(e)}")
import traceback
traceback.print_exc()
return {
"success": False,
"error": f"Core container communication failed: {str(e)}",
"message": f"Failed to fix NFO files: {str(e)}"
}
async def bulk_update_nfo_files(dependencies: dict, imdb_ids: list = None, fix_all: bool = False):
"""Update NFO files with missing dateadded elements from database"""
db = dependencies["db"]
config = dependencies["config"]
nfo_manager = dependencies["nfo_manager"]
if not config.manage_nfo:
return {
"success": False,
"message": "NFO management is disabled in configuration"
}
try:
# Build query based on parameters
if fix_all:
query = """
SELECT e.imdb_id, e.season, e.episode, e.aired, e.dateadded, e.source,
s.path as series_path
FROM episodes e
JOIN series s ON e.imdb_id = s.imdb_id
WHERE e.dateadded IS NOT NULL
AND e.has_video_file = TRUE
ORDER BY e.imdb_id, e.season, e.episode
"""
params = []
elif imdb_ids:
placeholders = ','.join(['%s'] * len(imdb_ids))
query = f"""
SELECT e.imdb_id, e.season, e.episode, e.aired, e.dateadded, e.source,
s.path as series_path
FROM episodes e
JOIN series s ON e.imdb_id = s.imdb_id
WHERE e.imdb_id IN ({placeholders})
AND e.dateadded IS NOT NULL
AND e.has_video_file = TRUE
ORDER BY e.imdb_id, e.season, e.episode
"""
params = imdb_ids
else:
return {
"success": False,
"message": "No IMDb IDs specified and fix_all not enabled"
}
# Get episodes to update
with db.get_connection() as conn:
cursor = conn.cursor()
cursor.execute(query, params)
episodes = [dict(row) for row in cursor.fetchall()]
if not episodes:
return {
"success": True,
"message": "No episodes found to update",
"updated_count": 0
}
# Update NFO files
updated_count = 0
errors = []
for ep in episodes:
try:
series_path = Path(ep['series_path'])
season_dir = series_path / config.tv_season_dir_format.format(season=ep['season'])
if not season_dir.exists():
continue
# Update NFO file
nfo_manager.create_episode_nfo(
season_dir=season_dir,
season_num=ep['season'],
episode_num=ep['episode'],
aired=ep['aired'],
dateadded=ep['dateadded'],
source=ep['source'],
lock_metadata=config.lock_metadata
)
updated_count += 1
print(f"✅ Updated NFO: {ep['imdb_id']} S{ep['season']:02d}E{ep['episode']:02d}")
except Exception as e:
error_msg = f"Failed to update {ep['imdb_id']} S{ep['season']:02d}E{ep['episode']:02d}: {str(e)}"
errors.append(error_msg)
print(f"{error_msg}")
return {
"success": True,
"message": f"Updated {updated_count} NFO files",
"updated_count": updated_count,
"total_episodes": len(episodes),
"errors": errors[:10] # Limit errors to first 10
}
except Exception as e:
return {
"success": False,
"error": str(e),
"message": f"Bulk NFO update failed: {str(e)}"
}
# Add database admin routes to the web interface
def register_database_admin_routes(app, dependencies):
"""Register database admin routes"""
@app.post("/api/admin/database/query")
async def api_database_query(query: str, limit: int = 100):
"""Execute a database query"""
return await execute_database_query(dependencies, query, limit)
@app.get("/api/admin/nfo/missing-dateadded")
async def api_episodes_missing_nfo_dateadded():
"""Get episodes missing dateadded in NFO files"""
return await get_episodes_missing_nfo_dateadded(dependencies)
@app.post("/api/admin/nfo/bulk-update")
async def api_bulk_update_nfo_files(imdb_ids: list = None, fix_all: bool = False):
"""Bulk update NFO files with missing dateadded via core container"""
# For now, we only support fix_all mode using the new core container approach
if fix_all or not imdb_ids:
return await fix_nfo_missing_dateadded(dependencies)
else:
# Legacy mode for specific IMDb IDs - fallback to old method
return await bulk_update_nfo_files(dependencies, imdb_ids, fix_all)
@app.get("/api/admin/database/quick-queries")
async def api_database_quick_queries():
"""Get predefined quick queries for common tasks"""
return {
"success": True,
"queries": [
{
"name": "Episodes missing dateadded in NFO",
"description": "Find episodes with dateadded in DB but missing from NFO files",
"query": "SELECT e.imdb_id, e.season, e.episode, e.dateadded, e.source, s.path FROM episodes e JOIN series s ON e.imdb_id = s.imdb_id WHERE e.dateadded IS NOT NULL AND e.has_video_file = TRUE ORDER BY e.last_updated DESC"
},
{
"name": "Recent webhook episodes",
"description": "Episodes processed in last 24 hours",
"query": "SELECT * FROM episodes WHERE last_updated > NOW() - INTERVAL '1 day' ORDER BY last_updated DESC"
},
{
"name": "Episodes by source type",
"description": "Count episodes by their date source",
"query": "SELECT source, COUNT(*) as count FROM episodes GROUP BY source ORDER BY count DESC"
},
{
"name": "Series with most episodes",
"description": "Series sorted by episode count",
"query": "SELECT s.imdb_id, s.path, COUNT(e.episode) as episode_count FROM series s LEFT JOIN episodes e ON s.imdb_id = e.imdb_id GROUP BY s.imdb_id, s.path ORDER BY episode_count DESC"
},
{
"name": "Episodes without video files",
"description": "Episodes in database without video files",
"query": "SELECT imdb_id, season, episode, dateadded, source FROM episodes WHERE has_video_file = FALSE ORDER BY imdb_id, season, episode"
}
]
}
+10 -6
View File
@@ -222,7 +222,7 @@ class SonarrClient:
import_date = earliest_import["date"]
_log("INFO", f"Found import date: {import_date} for episode {episode_id}")
# Check if this looks like an upgrade by comparing to renames
# Check chronological order of events
if rename_events:
earliest_rename = min(rename_events, key=lambda x: x["date"])
rename_date = earliest_rename["date"]
@@ -230,12 +230,16 @@ class SonarrClient:
try:
import_dt = datetime.fromisoformat(import_date.replace("Z", "+00:00"))
rename_dt = datetime.fromisoformat(rename_date.replace("Z", "+00:00"))
days_diff = (import_dt - rename_dt).days
# If import is significantly after rename, prefer rename date
if days_diff > 30:
_log("WARNING", f"Import {import_date} is {days_diff} days after rename {rename_date} - using rename date")
return rename_date
# If import happened BEFORE rename, it's valid original import
if import_dt <= rename_dt:
_log("INFO", f"Import {import_date} happened before/during rename {rename_date} - using import date")
return import_date
# If rename happened BEFORE import - always use aired date fallback
else:
_log("WARNING", f"Rename {rename_date} happened before import {import_date} - using aired date fallback")
return None # Trigger aired date fallback
except Exception as e:
_log("DEBUG", f"Error comparing dates: {e}")
+1
View File
@@ -61,6 +61,7 @@ class NFOGuardConfig:
# Batching and performance
self.batch_delay = self._get_float_env("BATCH_DELAY", 5.0, 0.1, 300.0)
self.max_concurrent = self._get_int_env("MAX_CONCURRENT_SERIES", 3, 1, 10)
self.sequential_delay = self._get_float_env("SEQUENTIAL_DELAY", 20.0, 0.0, 60.0) # Delay between sequential episodes (default 20s)
# Database
self.db_type = os.environ.get("DB_TYPE", "sqlite").lower()
+34 -1
View File
@@ -441,6 +441,29 @@ class NFOGuardDatabase:
return deleted_count > 0
def delete_series(self, imdb_id: str) -> bool:
"""
Delete a specific series from the database
Args:
imdb_id: Series IMDb ID
Returns:
True if series was deleted, False if not found
"""
with self.get_connection() as conn:
cursor = conn.cursor()
cursor.execute("""
DELETE FROM series
WHERE imdb_id = %s
""", (imdb_id,))
deleted_count = cursor.rowcount
conn.commit()
return deleted_count > 0
def delete_orphaned_movies(self) -> List[Dict]:
"""
Find and delete movies that don't have corresponding video files on disk
@@ -564,7 +587,17 @@ class NFOGuardDatabase:
"""Close all database connections"""
if hasattr(self._local, 'connection'):
try:
# For PostgreSQL, ensure all transactions are committed/rolled back
try:
# Force rollback any open transactions
self._local.connection.rollback()
except Exception:
pass
# Close the connection
self._local.connection.close()
delattr(self._local, 'connection')
except Exception:
print("✅ Database connection closed successfully")
except Exception as e:
print(f"⚠️ Error closing database connection: {e}")
pass # Connection may already be closed
+1 -1
View File
@@ -220,7 +220,7 @@ class EpisodeNFOManager:
lockdata_elem = ET.SubElement(episode_elem, "lockdata")
lockdata_elem.text = "true"
# Add comment with source
# Add comment with source at the bottom
comment = ET.Comment(f" Created by {self.manager_brand} - Source: {source} ")
episode_elem.append(comment)
+22 -41
View File
@@ -342,10 +342,6 @@ class NFOManager:
# This ensures they appear as a group at the bottom of the file
nfoguard_elements = []
# Add NFOGuard comment marker as the first of our additions
nfoguard_comment = ET.Comment(f" NFOGuard - Source: {source} ")
nfoguard_elements.append(nfoguard_comment)
# Add IMDb uniqueid
uniqueid = ET.Element("uniqueid", type="imdb", default="true")
uniqueid.text = imdb_id
@@ -384,6 +380,10 @@ class NFOManager:
lockdata.text = "true"
nfoguard_elements.append(lockdata)
# Add NFOGuard comment as the very last element (appears at bottom)
nfoguard_comment = ET.Comment(f" NFOGuard - Source: {source} ")
nfoguard_elements.append(nfoguard_comment)
# Now append all NFOGuard elements to the movie in one batch
# This ensures they appear as a contiguous block at the bottom
for elem in nfoguard_elements:
@@ -395,7 +395,7 @@ class NFOManager:
tree = ET.ElementTree(movie)
ET.indent(tree, space=" ", level=0)
# Write directly to file (comment is already embedded in XML)
# Write directly to file (comment is already embedded in XML at bottom)
with open(nfo_path, 'w', encoding='utf-8') as f:
f.write('<?xml version="1.0" encoding="utf-8"?>\n')
tree.write(f, encoding='unicode', xml_declaration=False)
@@ -455,22 +455,14 @@ class NFOManager:
lockdata = ET.SubElement(tvshow, "lockdata")
lockdata.text = "true"
# Add NFOGuard comment at the beginning
comment_text = f" Created by {self.manager_brand} "
# Add NFOGuard comment at the bottom
comment = ET.Comment(f" Created by {self.manager_brand} ")
tvshow.append(comment)
# Write file with proper formatting
tree = ET.ElementTree(tvshow)
ET.indent(tree, space=" ", level=0)
# Write to string first to add comment
xml_str = ET.tostring(tvshow, encoding='unicode')
# Add XML declaration and comment
full_xml = f'<?xml version="1.0" encoding="utf-8"?>\n<!--{comment_text}-->\n{xml_str}'
# Write to file
with open(nfo_path, 'w', encoding='utf-8') as f:
f.write(full_xml)
tree.write(nfo_path, encoding='utf-8', xml_declaration=True)
print(f"✅ Successfully created/updated TV show NFO: {nfo_path}")
print(f" IMDb ID: {imdb_id}" + (f", TVDB ID: {tvdb_id}" if tvdb_id else ""))
@@ -517,22 +509,14 @@ class NFOManager:
lockdata = ET.SubElement(season, "lockdata")
lockdata.text = "true"
# Add NFOGuard comment at the beginning
comment_text = f" Created by {self.manager_brand} "
# Add NFOGuard comment at the bottom
comment = ET.Comment(f" Created by {self.manager_brand} ")
season.append(comment)
# Write file with proper formatting
tree = ET.ElementTree(season)
ET.indent(tree, space=" ", level=0)
# Write to string first to add comment
xml_str = ET.tostring(season, encoding='unicode')
# Add XML declaration and comment
full_xml = f'<?xml version="1.0" encoding="utf-8"?>\n<!--{comment_text}-->\n{xml_str}'
# Write to file
with open(nfo_path, 'w', encoding='utf-8') as f:
f.write(full_xml)
tree.write(nfo_path, encoding='utf-8', xml_declaration=True)
print(f"✅ Successfully created/updated season NFO: {nfo_path}")
print(f" Season: {season_number}")
@@ -707,32 +691,29 @@ class NFOManager:
aired_str = str(aired)
aired_elem.text = aired_str[:10] if len(aired_str) >= 10 else aired_str
# Debug logging for dateadded
print(f"🔍 DEBUG: dateadded value: {repr(dateadded)} (type: {type(dateadded)})")
if dateadded:
dateadded_elem = ET.SubElement(episode, "dateadded")
# Convert datetime objects to strings
dateadded_elem.text = str(dateadded)
print(f"✅ Added dateadded to episode NFO: {dateadded}")
else:
print(f"❌ dateadded is empty/None, not adding to episode NFO")
# Add lockdata at the very end
if lock_metadata:
lockdata = ET.SubElement(episode, "lockdata")
lockdata.text = "true"
# Add NFOGuard comment at the beginning
comment_text = f" Created by {self.manager_brand} - Source: {source} "
# Add NFOGuard comment at the bottom
comment = ET.Comment(f" Created by {self.manager_brand} - Source: {source} ")
episode.append(comment)
# Write file with proper formatting
tree = ET.ElementTree(episode)
ET.indent(tree, space=" ", level=0)
# Write to string first to add comment
xml_str = ET.tostring(episode, encoding='unicode')
# Add XML declaration and comment
full_xml = f'<?xml version="1.0" encoding="utf-8"?>\n<!--{comment_text}-->\n{xml_str}'
# Write to file
with open(nfo_path, 'w', encoding='utf-8') as f:
f.write(full_xml)
tree.write(nfo_path, encoding='utf-8', xml_declaration=True)
print(f"✅ Successfully created/updated episode NFO: {nfo_path}")
print(f" S{season_num:02d}E{episode_num:02d}, Aired: {aired}, Date Added: {dateadded}")
+1 -1
View File
@@ -88,7 +88,7 @@ services:
nfoguard:
condition: service_healthy
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8081/"]
test: ["CMD", "curl", "-f", "http://localhost:8081/health"]
interval: 30s
timeout: 10s
retries: 3
+12
View File
@@ -146,6 +146,18 @@ def signal_handler(signum, frame):
_log("WARNING", f"Error closing database: {e}")
_log("INFO", "Graceful shutdown complete")
# Force exit after 2 seconds if graceful shutdown doesn't work
import threading
def force_exit():
import time
time.sleep(2)
_log("WARNING", "Force exiting after timeout")
os._exit(0)
force_thread = threading.Thread(target=force_exit, daemon=True)
force_thread.start()
sys.exit(0)
+238 -315
View File
@@ -96,11 +96,11 @@ async def get_movies_list(dependencies: dict,
params.append(source_filter)
if search:
where_conditions.append("(imdb_id LIKE %s OR path LIKE %s)")
where_conditions.append("(imdb_id ILIKE %s OR path ILIKE %s)")
params.extend([f"%{search}%", f"%{search}%"])
if imdb_search:
where_conditions.append("imdb_id LIKE %s")
where_conditions.append("imdb_id ILIKE %s")
params.append(f"%{imdb_search}%")
where_clause = " AND ".join(where_conditions) if where_conditions else "1=1"
@@ -165,11 +165,11 @@ async def get_tv_series_list(dependencies: dict,
having_conditions = []
if search:
where_conditions.append("(s.imdb_id LIKE %s OR s.path LIKE %s)")
where_conditions.append("(s.imdb_id ILIKE %s OR s.path ILIKE %s)")
params.extend([f"%{search}%", f"%{search}%"])
if imdb_search:
where_conditions.append("s.imdb_id LIKE %s")
where_conditions.append("s.imdb_id ILIKE %s")
params.append(f"%{imdb_search}%")
if source_filter:
@@ -600,56 +600,26 @@ async def update_episode_date(dependencies: dict, imdb_id: str, season: int, epi
has_video_file=episode_data.get('has_video_file', False)
)
# Create/update NFO file with new data
nfo_manager = dependencies["nfo_manager"]
config = dependencies["config"]
if config.manage_nfo:
try:
# Find the series directory based on IMDb ID
series_path = None
for tv_path in config.tv_paths:
for series_dir in Path(tv_path).iterdir():
if series_dir.is_dir() and imdb_id.lower() in series_dir.name.lower():
series_path = series_dir
break
if series_path:
break
if series_path:
season_dir = series_path / config.tv_season_dir_format.format(season=season)
if season_dir.exists():
nfo_manager.create_episode_nfo(
season_dir=season_dir,
season_num=season,
episode_num=episode,
aired=episode_data.get('aired'),
dateadded=dateadded,
source=source,
lock_metadata=config.lock_metadata
)
print(f"✅ Updated NFO file for {imdb_id} S{season:02d}E{episode:02d}")
else:
print(f"⚠️ Season directory not found: {season_dir}")
else:
print(f"⚠️ Series directory not found for {imdb_id}")
except Exception as e:
print(f"❌ Error updating NFO file for {imdb_id} S{season:02d}E{episode:02d}: {e}")
# NFO file management is handled by the core container
# Web interface only updates database entries
# Add to processing history
db.add_processing_history(
imdb_id=imdb_id,
media_type="episode",
event_type="manual_date_update",
details={
"season": season,
"episode": episode,
"old_source": episode_data.get('source'),
"new_source": source,
"dateadded": dateadded
}
)
try:
db.add_processing_history(
imdb_id=imdb_id,
media_type="episode",
event_type="manual_date_update",
details={
"season": season,
"episode": episode,
"old_source": episode_data.get('source'),
"new_source": source,
"dateadded": dateadded
}
)
except Exception as e:
print(f"⚠️ Failed to add processing history: {e}")
# Don't fail the entire update for history logging issues
return {"status": "success", "message": f"Updated episode {imdb_id} S{season:02d}E{episode:02d}"}
@@ -882,7 +852,7 @@ async def get_movie_date_options(dependencies: dict, imdb_id: str):
async def get_episode_date_options(dependencies: dict, imdb_id: str, season: int, episode: int):
"""Get available date options for an episode"""
"""Get available date options for an episode (simplified for web interface)"""
print(f"🔍 DEBUG: get_episode_date_options called with imdb_id={imdb_id}, season={season}, episode={episode}")
db = dependencies["db"]
@@ -936,267 +906,7 @@ async def get_episode_date_options(dependencies: dict, imdb_id: str, season: int
"description": f"Use original air date: {episode_data['aired']}"
})
# Option 3: Active lookup from external sources
try:
# Get TV processor and clients from dependencies
tv_processor = dependencies.get("tv_processor")
print(f"🔍 DEBUG: tv_processor available: {tv_processor is not None}")
if tv_processor:
print(f"🔍 DEBUG: tv_processor has external_clients: {hasattr(tv_processor, 'external_clients')}")
print(f"🔍 DEBUG: tv_processor has sonarr: {hasattr(tv_processor, 'sonarr')}")
if tv_processor and hasattr(tv_processor, 'external_clients'):
external_clients = tv_processor.external_clients
print(f"🔍 DEBUG: external_clients available: {external_clients is not None}")
if external_clients:
print(f"🔍 DEBUG: TMDB enabled: {external_clients.tmdb.enabled if hasattr(external_clients, 'tmdb') else 'No TMDB client'}")
# Check Sonarr for import dates
if tv_processor.sonarr and tv_processor.sonarr.enabled:
try:
print(f"🔍 DEBUG: Attempting Sonarr lookup for {imdb_id}")
# Look up the series and episode in Sonarr
series_data = tv_processor.sonarr.series_by_imdb(imdb_id)
# If IMDb lookup fails, try direct series lookup as fallback
if not series_data:
print(f"🔍 DEBUG: IMDb lookup failed, trying direct series lookup")
try:
# Let's also debug what series are available
all_series = tv_processor.sonarr.get_all_series()
print(f"🔍 DEBUG: Found {len(all_series)} total series in Sonarr")
# Look for Lincoln Lawyer specifically
lincoln_series = [s for s in all_series if 'lincoln' in s.get('title', '').lower()]
print(f"🔍 DEBUG: Lincoln Lawyer series found: {len(lincoln_series)}")
for ls in lincoln_series:
print(f" - Title: '{ls.get('title')}', IMDb: '{ls.get('imdbId')}', ID: {ls.get('id')}")
# Try direct lookup first
series_data = tv_processor.sonarr.series_by_imdb_direct(imdb_id)
# If still no match but we found Lincoln Lawyer series, try fuzzy matching
if not series_data and lincoln_series:
target_imdb_num = imdb_id.replace('tt', '').lower()
print(f"🔍 DEBUG: Trying fuzzy match for IMDb number: {target_imdb_num}")
for ls in lincoln_series:
ls_imdb = ls.get('imdbId', '')
ls_imdb_num = ls_imdb.replace('tt', '').lower()
print(f" - Comparing {target_imdb_num} vs {ls_imdb_num}")
# Check if IMDb numbers are close (within 10 digits)
if ls_imdb_num and target_imdb_num:
try:
target_num = int(target_imdb_num)
ls_num = int(ls_imdb_num)
diff = abs(target_num - ls_num)
print(f" - Numeric difference: {diff}")
if diff <= 10: # Allow small IMDb ID differences
print(f"✅ Found close IMDb match: {ls_imdb} vs {imdb_id} (diff: {diff})")
series_data = ls
break
except ValueError:
continue
except Exception as e:
print(f"⚠️ Direct series lookup also failed: {e}")
import traceback
print(f" Traceback: {traceback.format_exc()}")
print(f"🔍 DEBUG: Series data found: {series_data is not None}")
if series_data:
series_id = series_data.get('id')
series_title = series_data.get('title', 'Unknown')
print(f"🔍 DEBUG: Found series '{series_title}' with ID {series_id}")
if series_id:
# Get episodes for the series
print(f"🔍 DEBUG: Getting episodes for series {series_id}")
episodes = tv_processor.sonarr.episodes_for_series(series_id)
print(f"🔍 DEBUG: Found {len(episodes)} episodes")
for ep in episodes:
ep_season = ep.get('seasonNumber')
ep_episode = ep.get('episodeNumber')
# Convert to int for proper comparison (handle both string and int from Sonarr)
try:
ep_season = int(ep_season) if ep_season is not None else None
ep_episode = int(ep_episode) if ep_episode is not None else None
except (ValueError, TypeError):
continue # Skip episodes with invalid season/episode numbers
if ep_season == season and ep_episode == episode:
episode_id = ep.get('id')
ep_title = ep.get('title', 'Unknown')
ep_air_date = ep.get('airDate') # Get air date from Sonarr
print(f"🔍 DEBUG: Found target episode '{ep_title}' with ID {episode_id}, airDate: {ep_air_date}")
if episode_id:
# Get import history for this specific episode
print(f"🔍 DEBUG: Getting import history for episode {episode_id}")
import_date = tv_processor.sonarr.get_episode_import_history(episode_id)
print(f"🔍 DEBUG: Import date found: {import_date}")
if import_date:
# Check if this is different from current date
current_dateadded = episode_data.get('dateadded')
current_date_str = current_dateadded.strftime('%Y-%m-%d') if current_dateadded else ''
if not current_dateadded or not current_date_str.startswith(import_date[:10]):
options.append({
"type": "sonarr_import",
"label": "Sonarr Import Date",
"date": import_date,
"source": "sonarr:import_history",
"description": f"Import date from Sonarr: {import_date[:10]}"
})
print(f"✅ Added Sonarr import option: {import_date[:10]}")
# If no import date but we have air date from Sonarr, add as air date option
if not import_date and ep_air_date:
current_aired = episode_data.get('aired', '')
current_dateadded = episode_data.get('dateadded', '')
# Add air date option if different from current or missing
if not current_aired or current_aired != ep_air_date:
options.append({
"type": "sonarr_air",
"label": "Sonarr Air Date",
"date": f"{ep_air_date}T20:00:00",
"source": "sonarr:airdate",
"description": f"Air date from Sonarr: {ep_air_date}"
})
print(f"✅ Added Sonarr air date option: {ep_air_date}")
# If no dateadded, suggest using air date as import date fallback
if not current_dateadded:
options.append({
"type": "sonarr_air_fallback",
"label": "Use Air Date as Import Date",
"date": f"{ep_air_date}T20:00:00",
"source": "sonarr:aired_fallback",
"description": f"Use Sonarr air date as import date: {ep_air_date}"
})
print(f"✅ Added Sonarr air date fallback option: {ep_air_date}")
break
else:
print(f"❌ No series found in Sonarr for {imdb_id}")
except Exception as e:
print(f"⚠️ Failed to get Sonarr import date for {imdb_id} S{season:02d}E{episode:02d}: {e}")
import traceback
print(f" Traceback: {traceback.format_exc()}")
# Check TMDB for episode air dates
if external_clients.tmdb.enabled:
try:
print(f"🔍 DEBUG: Attempting TMDB lookup for {imdb_id}")
# Get TMDB TV series ID from IMDb ID using find endpoint
tv_find_result = external_clients.tmdb._get(f"/find/{imdb_id}", {"external_source": "imdb_id"})
print(f"🔍 DEBUG: TMDB find result: {tv_find_result is not None}")
print(f"🔍 DEBUG: TMDB raw response: {tv_find_result}")
# Check both tv_results and tv_episode_results
tmdb_id = None
tv_title = "Unknown"
if tv_find_result and tv_find_result.get("tv_results"):
tv_results = tv_find_result.get("tv_results", [])
print(f"🔍 DEBUG: Found {len(tv_results)} TV results")
if tv_results:
tv_show = tv_results[0]
tmdb_id = tv_show.get("id")
tv_title = tv_show.get("name", "Unknown")
print(f"🔍 DEBUG: Found TMDB series '{tv_title}' with ID {tmdb_id}")
# Fallback: Check tv_episode_results for show_id
elif tv_find_result and tv_find_result.get("tv_episode_results"):
episode_results = tv_find_result.get("tv_episode_results", [])
print(f"🔍 DEBUG: Found {len(episode_results)} TV episode results")
if episode_results:
tmdb_episode_data = episode_results[0]
tmdb_id = tmdb_episode_data.get("show_id")
episode_name = tmdb_episode_data.get("name", "Unknown")
print(f"🔍 DEBUG: Found TMDB series via episode '{episode_name}' with show_id {tmdb_id}")
if tmdb_id:
print(f"🔍 DEBUG: Using TMDB ID {tmdb_id} for series lookup")
# Get episode air date from TMDB
print(f"🔍 DEBUG: Getting TMDB season {season} episodes for series {tmdb_id}")
episodes = external_clients.tmdb.get_tv_season_episodes(tmdb_id, season)
print(f"🔍 DEBUG: TMDB episodes found: {episodes}")
if episode in episodes:
air_date = episodes[episode]
print(f"🔍 DEBUG: TMDB air date for S{season:02d}E{episode:02d}: {air_date}")
if air_date:
# Check if this is different from current aired date
current_aired = episode_data.get('aired', '')
if not current_aired or current_aired != air_date:
options.append({
"type": "tmdb_air",
"label": "TMDB Air Date",
"date": f"{air_date}T20:00:00", # Default to 8 PM
"source": "tmdb:airdate",
"description": f"Air date from TMDB: {air_date}"
})
print(f"✅ Added TMDB air date option: {air_date}")
# If no aired date in database, also add this as "Use Air Date" option
if not current_aired:
options.insert(1, { # Insert after current option
"type": "airdate_tmdb",
"label": "Use Air Date (TMDB)",
"date": f"{air_date}T20:00:00",
"source": "airdate",
"description": f"Use air date from TMDB: {air_date}"
})
print(f"✅ Added 'Use Air Date' option from TMDB: {air_date}")
else:
print(f"❌ Episode {episode} not found in TMDB season {season} data")
else:
print(f"❌ No TV series ID found in TMDB for {imdb_id}")
except Exception as e:
print(f"⚠️ Failed to get TMDB air date for {imdb_id} S{season:02d}E{episode:02d}: {e}")
import traceback
print(f" Traceback: {traceback.format_exc()}")
# Check external clients for episode air dates (TVDB, OMDb)
if hasattr(external_clients, 'get_episode_air_date'):
try:
air_date = external_clients.get_episode_air_date(imdb_id, season, episode)
if air_date:
# Check if this is different from current aired date
current_aired = episode_data.get('aired', '')
if not current_aired or current_aired != air_date:
options.append({
"type": "external_air",
"label": "External Air Date",
"date": f"{air_date}T20:00:00", # Default to 8 PM
"source": "external:airdate",
"description": f"Air date from external sources: {air_date}"
})
# If no aired date in database and not already added from TMDB, add this as "Use Air Date" option
if not current_aired and not any(opt.get('type') == 'airdate_tmdb' for opt in options):
options.insert(1, { # Insert after current option
"type": "airdate_external",
"label": "Use Air Date (External)",
"date": f"{air_date}T20:00:00",
"source": "airdate",
"description": f"Use air date from external sources: {air_date}"
})
except Exception as e:
print(f"⚠️ Failed to get external air date for {imdb_id} S{season:02d}E{episode:02d}: {e}")
except Exception as e:
print(f"⚠️ External source lookup failed for {imdb_id} S{season:02d}E{episode:02d}: {e}")
# Option 4: Manual entry
# Option 3: Manual entry
options.append({
"type": "manual",
"label": "Manual Entry",
@@ -1216,4 +926,217 @@ async def get_episode_date_options(dependencies: dict, imdb_id: str, season: int
"episode": episode,
"current_data": episode_data,
"options": options
}
}
async def delete_episode(dependencies: dict, imdb_id: str, season: int, episode: int):
"""Delete an episode from the database"""
db = dependencies["db"]
# Check if episode exists
episode_data = db.get_episode_date(imdb_id, season, episode)
if not episode_data:
raise HTTPException(status_code=404, detail="Episode not found")
# Delete from database
try:
with db.get_connection() as conn:
cursor = conn.cursor()
cursor.execute(
"DELETE FROM episodes WHERE imdb_id = %s AND season = %s AND episode = %s",
(imdb_id, season, episode)
)
conn.commit()
# Add to processing history
try:
db.add_processing_history(
imdb_id=imdb_id,
media_type="episode",
event_type="manual_deletion",
details={
"season": season,
"episode": episode,
"deleted_source": episode_data.get('source'),
"deleted_dateadded": episode_data.get('dateadded')
}
)
except Exception as e:
print(f"⚠️ Failed to add processing history: {e}")
return {"success": True, "status": "success", "message": f"Deleted episode {imdb_id} S{season:02d}E{episode:02d}"}
except Exception as e:
print(f"❌ Error deleting episode: {e}")
raise HTTPException(status_code=500, detail=f"Failed to delete episode: {str(e)}")
async def delete_movie(dependencies: dict, imdb_id: str):
"""Delete a movie from the database"""
db = dependencies["db"]
# Check if movie exists
movie_data = db.get_movie_dates(imdb_id)
if not movie_data:
raise HTTPException(status_code=404, detail="Movie not found")
# Delete from database
try:
with db.get_connection() as conn:
cursor = conn.cursor()
cursor.execute("DELETE FROM movies WHERE imdb_id = %s", (imdb_id,))
conn.commit()
# Add to processing history
try:
db.add_processing_history(
imdb_id=imdb_id,
media_type="movie",
event_type="manual_deletion",
details={
"deleted_source": movie_data.get('source'),
"deleted_dateadded": movie_data.get('dateadded'),
"deleted_path": movie_data.get('path')
}
)
except Exception as e:
print(f"⚠️ Failed to add processing history: {e}")
return {"success": True, "status": "success", "message": f"Deleted movie {imdb_id}"}
except Exception as e:
print(f"❌ Error deleting movie: {e}")
raise HTTPException(status_code=500, detail=f"Failed to delete movie: {str(e)}")
def register_web_routes(app, dependencies):
"""Register all web API routes with FastAPI app"""
from fastapi import Request, Response
# Dashboard and stats endpoints
@app.get("/api/dashboard")
async def api_dashboard():
return await get_dashboard_stats(dependencies)
@app.get("/api/dashboard/stats")
async def api_dashboard_stats():
return await get_dashboard_stats(dependencies)
# Movies endpoints
@app.get("/api/movies")
async def api_movies_list(skip: int = 0, limit: int = 100, has_date: bool = None,
source_filter: str = None, search: str = None, imdb_search: str = None):
return await get_movies_list(dependencies, skip, limit, has_date, source_filter, search, imdb_search)
@app.post("/api/movies/{imdb_id}/update-date")
async def api_update_movie_date(imdb_id: str, dateadded: str = None, source: str = "manual"):
return await update_movie_date(dependencies, imdb_id, dateadded, source)
@app.put("/api/movies/{imdb_id}")
async def api_update_movie(imdb_id: str, dateadded: str = None, source: str = "manual"):
return await update_movie_date(dependencies, imdb_id, dateadded, source)
@app.get("/api/movies/{imdb_id}/date-options")
async def api_movie_date_options(imdb_id: str):
return await get_movie_date_options(dependencies, imdb_id)
# TV series endpoints
@app.get("/api/series")
async def api_series_list(skip: int = 0, limit: int = 50, search: str = None,
imdb_search: str = None, date_filter: str = None, source_filter: str = None):
return await get_tv_series_list(dependencies, skip, limit, search, imdb_search, date_filter, source_filter)
@app.get("/api/series/{imdb_id}/episodes")
async def api_series_episodes(imdb_id: str):
return await get_series_episodes(dependencies, imdb_id)
@app.get("/api/series/sources")
async def api_series_sources():
return await get_series_sources(dependencies)
@app.get("/api/series/debug/date-distribution")
async def api_debug_series_date_distribution():
return await debug_series_date_distribution(dependencies)
# Episode endpoints - WORKING VERSIONS
@app.post("/api/episodes/{imdb_id}/{season}/{episode}/update-date")
async def api_update_episode_date(imdb_id: str, season: int, episode: int,
dateadded: str = None, source: str = "manual"):
return await update_episode_date(dependencies, imdb_id, season, episode, dateadded, source)
@app.put("/api/episodes/{imdb_id}/{season}/{episode}")
async def api_update_episode(imdb_id: str, season: int, episode: int,
dateadded: str = None, source: str = "manual"):
return await update_episode_date(dependencies, imdb_id, season, episode, dateadded, source)
# Register DELETE route explicitly
async def api_delete_episode_handler(imdb_id: str, season: int, episode: int):
return await delete_episode(dependencies, imdb_id, season, episode)
app.add_api_route(
"/api/episodes/{imdb_id}/{season}/{episode}",
api_delete_episode_handler,
methods=["DELETE"],
name="delete_episode"
)
print("✅ DELETE /api/episodes/{imdb_id}/{season}/{episode} route registered")
@app.get("/api/episodes/{imdb_id}/{season}/{episode}/date-options")
async def api_episode_date_options(imdb_id: str, season: int, episode: int):
return await get_episode_date_options(dependencies, imdb_id, season, episode)
# Movie deletion endpoint
@app.delete("/api/movies/{imdb_id}")
async def api_delete_movie(imdb_id: str):
return await delete_movie(dependencies, imdb_id)
# Bulk operations
@app.post("/api/bulk/update-source")
async def api_bulk_update_source(media_type: str, old_source: str, new_source: str):
return await bulk_update_source(dependencies, media_type, old_source, new_source)
# Reports
@app.get("/api/reports/missing-dates")
async def api_missing_dates_report():
return await get_missing_dates_report(dependencies)
# Authentication endpoints (for web interface compatibility)
@app.get("/api/auth/status")
async def api_auth_status(request: Request):
"""Check authentication status"""
auth_enabled = dependencies.get("auth_enabled", False)
if not auth_enabled:
return {"authenticated": True, "auth_enabled": False, "message": "Authentication disabled"}
session_manager = dependencies.get("session_manager")
if not session_manager:
return {"authenticated": False, "auth_enabled": True, "message": "Session manager not available"}
session_token = request.cookies.get("nfoguard_session")
if session_token:
username = session_manager.get_session_user(session_token)
if username:
return {"authenticated": True, "auth_enabled": True, "username": username}
return {"authenticated": False, "auth_enabled": True, "message": "Not authenticated"}
@app.post("/api/auth/logout")
async def api_auth_logout(request: Request, response: Response):
"""Logout endpoint - clears session"""
session_manager = dependencies.get("session_manager")
if session_manager:
session_token = request.cookies.get("nfoguard_session")
if session_token:
session_manager.delete_session(session_token)
response.delete_cookie("nfoguard_session")
return {"status": "logged_out", "message": "Session cleared"}
# Health endpoint
@app.get("/health")
async def health_check():
"""Health check endpoint for container monitoring"""
return {"status": "healthy", "service": "nfoguard-web"}
print("✅ Web routes registered successfully")
+147
View File
@@ -225,6 +225,153 @@ class WebDatabase:
"""
return self.execute_query(query)
# Episode-specific methods for web interface
def get_episode_date(self, imdb_id: str, season: int, episode: int) -> Optional[Dict]:
"""Get episode data including dates"""
query = """
SELECT imdb_id, season, episode, aired, dateadded, source, has_video_file, last_updated
FROM episodes
WHERE imdb_id = %s AND season = %s AND episode = %s
"""
return self.execute_single(query, (imdb_id, season, episode))
def upsert_episode_date(self, imdb_id: str, season: int, episode: int,
aired: Optional[str], dateadded: Optional[str],
source: str, has_video_file: bool = False) -> None:
"""Update or insert episode date information"""
# First check if episode exists
existing = self.get_episode_date(imdb_id, season, episode)
# Temporarily disable autocommit for the transaction
original_autocommit = self.connection.autocommit
self.connection.autocommit = False
try:
with self.connection.cursor() as cursor:
if existing:
# Update existing episode
query = """
UPDATE episodes
SET aired = %s, dateadded = %s, source = %s, has_video_file = %s, last_updated = NOW()
WHERE imdb_id = %s AND season = %s AND episode = %s
"""
cursor.execute(query, (aired, dateadded, source, has_video_file, imdb_id, season, episode))
else:
# Insert new episode
query = """
INSERT INTO episodes (imdb_id, season, episode, aired, dateadded, source, has_video_file, last_updated)
VALUES (%s, %s, %s, %s, %s, %s, %s, NOW())
"""
cursor.execute(query, (imdb_id, season, episode, aired, dateadded, source, has_video_file))
self.connection.commit()
except Exception as e:
self.connection.rollback()
logger.error(f"Failed to upsert episode date: {e}")
raise
finally:
# Restore original autocommit setting
self.connection.autocommit = original_autocommit
def get_movie_dates(self, imdb_id: str) -> Optional[Dict]:
"""Get movie data including dates"""
query = """
SELECT imdb_id, path, released, dateadded, source, has_video_file, last_updated
FROM movies
WHERE imdb_id = %s
"""
return self.execute_single(query, (imdb_id,))
def upsert_movie_dates(self, imdb_id: str, released: Optional[str],
dateadded: Optional[str], source: str,
has_video_file: bool = False, path: str = "") -> None:
"""Update or insert movie date information"""
# First check if movie exists
existing = self.get_movie_dates(imdb_id)
# Temporarily disable autocommit for the transaction
original_autocommit = self.connection.autocommit
self.connection.autocommit = False
try:
with self.connection.cursor() as cursor:
if existing:
# Update existing movie
query = """
UPDATE movies
SET released = %s, dateadded = %s, source = %s, has_video_file = %s, last_updated = NOW()
WHERE imdb_id = %s
"""
cursor.execute(query, (released, dateadded, source, has_video_file, imdb_id))
else:
# Insert new movie
query = """
INSERT INTO movies (imdb_id, path, released, dateadded, source, has_video_file, last_updated)
VALUES (%s, %s, %s, %s, %s, %s, NOW())
"""
cursor.execute(query, (imdb_id, path, released, dateadded, source, has_video_file))
self.connection.commit()
except Exception as e:
self.connection.rollback()
logger.error(f"Failed to upsert movie dates: {e}")
raise
finally:
# Restore original autocommit setting
self.connection.autocommit = original_autocommit
def get_connection(self):
"""Get database connection for advanced operations"""
return self.connection
def _get_first_value(self, row):
"""Extract first value from a database row (compatibility method)"""
if row is None:
return None
if isinstance(row, dict):
return list(row.values())[0] if row else None
return row[0] if row else None
def get_stats(self) -> Dict[str, Any]:
"""Get basic database statistics (compatibility method)"""
return self.get_dashboard_stats()
def add_processing_history(self, imdb_id: str, media_type: str, event_type: str, details: Dict) -> None:
"""Add processing history entry (simplified for web interface)"""
# Temporarily disable autocommit for the transaction
original_autocommit = self.connection.autocommit
self.connection.autocommit = False
try:
with self.connection.cursor() as cursor:
# Check if processing_history table exists
cursor.execute("""
SELECT EXISTS (
SELECT FROM information_schema.tables
WHERE table_name = 'processing_history'
)
""")
table_exists = cursor.fetchone()[0]
if table_exists:
query = """
INSERT INTO processing_history (imdb_id, media_type, event_type, details, processed_at)
VALUES (%s, %s, %s, %s, NOW())
"""
import json
cursor.execute(query, (imdb_id, media_type, event_type, json.dumps(details)))
self.connection.commit()
else:
# Table doesn't exist, skip logging
logger.debug("Processing history table not found, skipping log entry")
except Exception as e:
self.connection.rollback()
logger.error(f"Failed to add processing history: {e}")
# Don't raise, this is non-critical
finally:
# Restore original autocommit setting
self.connection.autocommit = original_autocommit
def close(self):
"""Close database connection"""
if self.connection:
+283 -1
View File
@@ -886,4 +886,286 @@ body {
.d-block { display: block; }
.d-flex { display: flex; }
.justify-content-between { justify-content: space-between; }
.align-items-center { align-items: center; }
.align-items-center { align-items: center; }
/* Database Admin Tools Styles */
.query-results {
margin-top: 1rem;
padding: 1rem;
background: #f8f9fa;
border: 1px solid var(--border-color);
border-radius: 0.375rem;
}
.query-info {
padding: 1rem;
border-radius: 0.375rem;
margin-bottom: 1rem;
font-family: 'Monaco', 'Menlo', 'Ubuntu Mono', monospace;
font-size: 0.875rem;
}
.query-info.success {
background-color: #d4edda;
border: 1px solid #c3e6cb;
color: #155724;
}
.query-info.error {
background-color: #f8d7da;
border: 1px solid #f5c6cb;
color: #721c24;
}
.results-table {
width: 100%;
border-collapse: collapse;
font-size: 0.875rem;
background: white;
border-radius: 0.375rem;
overflow: hidden;
box-shadow: var(--shadow);
}
.results-table th {
background-color: var(--dark-color);
color: white;
padding: 0.75rem;
text-align: left;
font-weight: 600;
}
.results-table td {
padding: 0.75rem;
border-bottom: 1px solid var(--border-color);
vertical-align: top;
}
.results-table tr:hover {
background-color: #f8f9fa;
}
.results-table td em {
color: var(--text-muted);
font-style: italic;
}
.query-results .table-container,
.admin-results .table-container {
max-height: 400px;
overflow-y: auto;
border: 1px solid var(--border-color);
border-radius: 0.375rem;
}
/* NFO Repair Table Styling */
.nfo-repair-table-container {
background: white;
border-radius: 0.5rem;
box-shadow: var(--shadow);
overflow-x: auto;
overflow-y: auto;
max-height: 600px;
margin-bottom: 1rem;
border: 1px solid var(--border-color);
}
.nfo-repair-table {
width: 100%;
min-width: 800px;
border-collapse: collapse;
font-size: 0.875rem;
background: white;
}
.nfo-repair-table th {
background-color: var(--dark-color);
color: white;
padding: 0.75rem 0.5rem;
text-align: left;
font-weight: 600;
position: sticky;
top: 0;
z-index: 10;
}
.nfo-repair-table td {
padding: 0.75rem 0.5rem;
border-bottom: 1px solid var(--border-color);
vertical-align: top;
}
.nfo-repair-table tr:hover {
background-color: #f8f9fa;
}
.media-type-badge {
padding: 0.25rem 0.5rem;
border-radius: 0.25rem;
font-size: 0.75rem;
font-weight: 600;
text-transform: uppercase;
}
.media-type-badge.episode {
background-color: #e3f2fd;
color: #1976d2;
}
.media-type-badge.movie {
background-color: #f3e5f5;
color: #7b1fa2;
}
.status-ready {
color: #2e7d32;
font-weight: 600;
font-size: 0.75rem;
}
.status-missing {
color: #d32f2f;
font-weight: 600;
font-size: 0.75rem;
}
.text-center {
text-align: center;
}
.text-muted {
color: #6c757d;
}
.nfo-fix-actions {
margin-top: 1rem;
padding: 1rem;
background-color: #f8f9fa;
border-radius: 0.5rem;
text-align: center;
}
.nfo-fix-actions .btn {
margin-bottom: 0.5rem;
}
.admin-tools {
display: flex;
flex-direction: column;
gap: 0.5rem;
}
.admin-tools .btn {
text-align: left;
justify-content: flex-start;
}
.admin-results {
margin-top: 1rem;
padding: 1rem;
background: #f8f9fa;
border: 1px solid var(--border-color);
border-radius: 0.375rem;
}
.scan-summary,
.fix-summary {
padding: 1rem;
border-radius: 0.375rem;
margin-bottom: 1rem;
}
.scan-summary {
background-color: #e7f3ff;
border: 1px solid #b3d9ff;
}
.fix-summary.success {
background-color: #d4edda;
border: 1px solid #c3e6cb;
color: #155724;
}
.fix-errors {
background-color: #fff3cd;
border: 1px solid #ffeaa7;
padding: 1rem;
border-radius: 0.375rem;
margin-top: 1rem;
}
.fix-errors ul {
margin: 0.5rem 0 0 1rem;
}
.missing-episodes {
margin-top: 1rem;
}
.loading {
text-align: center;
padding: 2rem;
color: var(--text-muted);
font-style: italic;
}
.loading::before {
content: "⏳ ";
}
.success {
background-color: #d4edda;
border: 1px solid #c3e6cb;
color: #155724;
padding: 1rem;
border-radius: 0.375rem;
margin: 1rem 0;
}
.error {
background-color: #f8d7da;
border: 1px solid #f5c6cb;
color: #721c24;
padding: 1rem;
border-radius: 0.375rem;
margin: 1rem 0;
}
/* Make text areas more distinctive */
textarea {
font-family: 'Monaco', 'Menlo', 'Ubuntu Mono', monospace;
font-size: 0.875rem;
resize: vertical;
}
/* Button variations for admin tools */
.btn.btn-info {
background-color: #17a2b8;
border-color: #17a2b8;
color: white;
}
.btn.btn-info:hover {
background-color: #138496;
border-color: #117a8b;
}
/* Responsive adjustments for admin tools */
@media (max-width: 768px) {
.table-container {
max-height: 300px;
}
.results-table {
font-size: 0.75rem;
}
.results-table th,
.results-table td {
padding: 0.5rem;
}
.admin-tools {
gap: 0.75rem;
}
}
+78 -2
View File
@@ -4,7 +4,7 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>NFOGuard - Database Management</title>
<link rel="stylesheet" href="/static/css/styles.css">
<link rel="stylesheet" href="/static/css/styles.css?v=2.8.2-20241026-nfo-table-redesign">
<link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0/css/all.min.css" rel="stylesheet">
</head>
<body>
@@ -390,6 +390,82 @@
<i class="fas fa-sync"></i> Refresh Stats
</button>
</div>
<!-- Database Query Tool -->
<div class="tool-card">
<h3><i class="fas fa-terminal"></i> Database Query Tool</h3>
<p>Execute custom SQL queries on the NFOGuard database</p>
<form id="database-query-form">
<div class="form-group">
<label>Quick Queries:</label>
<select id="quick-query-select" onchange="loadQuickQuery()">
<option value="">Select a predefined query...</option>
</select>
</div>
<div class="form-group">
<label>SQL Query:</label>
<textarea id="sql-query" rows="4" placeholder="SELECT * FROM episodes WHERE dateadded IS NULL LIMIT 10"></textarea>
<small>Query will be automatically limited to 100 results for safety</small>
</div>
<div class="form-group">
<label>Result Limit:</label>
<select id="query-limit">
<option value="10">10 rows</option>
<option value="50">50 rows</option>
<option value="100" selected>100 rows</option>
</select>
</div>
<button type="submit" class="btn btn-info">
<i class="fas fa-play"></i> Execute Query
</button>
</form>
<div id="query-results" class="query-results" style="display: none;">
<h4>Query Results:</h4>
<div id="query-results-content"></div>
</div>
</div>
<!-- NFO File Repair Tool -->
<div class="tool-card">
<h3><i class="fas fa-file-medical"></i> NFO File Repair</h3>
<p>Fix missing &lt;dateadded&gt; elements in NFO files</p>
<div class="form-group">
<button class="btn btn-warning" onclick="checkMissingNFODates()">
<i class="fas fa-search"></i> Scan for Missing dateadded in NFO Files
</button>
</div>
<div id="nfo-scan-results" style="display: none;">
<div id="nfo-scan-content"></div>
<div class="form-group">
<button class="btn btn-success" onclick="fixAllMissingNFODates()">
<i class="fas fa-wrench"></i> Fix All Missing dateadded Elements
</button>
<button class="btn btn-secondary" onclick="fixSpecificSeries()">
<i class="fas fa-cog"></i> Fix Specific Series
</button>
</div>
</div>
</div>
<!-- Database Admin Tools -->
<div class="tool-card">
<h3><i class="fas fa-user-shield"></i> Database Admin</h3>
<p>Advanced database management tools</p>
<div class="admin-tools">
<button class="btn btn-info" onclick="showEpisodesMissingNFODateadded()">
<i class="fas fa-exclamation-triangle"></i> Episodes Missing NFO dateadded
</button>
<button class="btn btn-secondary" onclick="exportDatabaseReport()">
<i class="fas fa-download"></i> Export Database Report
</button>
<button class="btn btn-secondary" onclick="showRecentWebhookActivity()">
<i class="fas fa-clock"></i> Recent Webhook Activity
</button>
</div>
<div id="admin-results" class="admin-results" style="display: none;">
<div id="admin-results-content"></div>
</div>
</div>
</div>
</div>
</main>
@@ -457,6 +533,6 @@
<!-- Toast Notifications -->
<div class="toast-container" id="toast-container"></div>
<script src="/static/js/app.js"></script>
<script src="/static/js/app.js?v=3.0.1-nfo-table-redesign"></script>
</body>
</html>
+479 -43
View File
@@ -92,6 +92,7 @@ function initializeEventListeners() {
async function apiCall(endpoint, options = {}) {
try {
const response = await fetch(endpoint, {
credentials: 'include', // Include cookies for authentication
headers: {
'Content-Type': 'application/json',
...options.headers
@@ -538,17 +539,17 @@ function showEpisodesModal(data) {
`<td>${dateadded}</td>`;
return `
<tr class="${rowClass}" data-has-date="${!missingDate}">
<tr class="${rowClass}" data-has-date="${!missingDate}" data-has-video="${episode.has_video_file}">
<td>S${episode.season.toString().padStart(2, '0')}E${episode.episode.toString().padStart(2, '0')}</td>
<td>${episode.aired || '-'}</td>
${dateCell}
<td><span class="badge badge-secondary">${episode.source_description || episode.source || 'Unknown'}</span></td>
<td>${hasVideoBadge}</td>
<td>
<button class="btn btn-sm btn-primary" onclick="editEpisode('${data.series.imdb_id}', ${episode.season}, ${episode.episode}, '${dateadded}', '${episode.source || ''}')">
<button class="btn btn-sm btn-primary" onclick="editEpisode('${data.series.imdb_id}', ${parseInt(episode.season)}, ${parseInt(episode.episode)}, '${dateadded}', '${episode.source || ''}')">
<i class="fas fa-edit"></i> Edit
</button>
<button class="btn btn-sm btn-danger" onclick="deleteEpisode('${data.series.imdb_id}', ${episode.season}, ${episode.episode})" style="margin-left: 5px;">
<button class="btn btn-sm btn-danger" onclick="deleteEpisode('${data.series.imdb_id}', ${parseInt(episode.season)}, ${parseInt(episode.episode)})" style="margin-left: 5px;">
<i class="fas fa-trash"></i> Delete
</button>
</td>
@@ -649,7 +650,7 @@ function updateReportTables(data) {
<td>${episode.aired || '-'}</td>
<td><span class="badge badge-warning">${episode.source_description || episode.source || 'Unknown'}</span></td>
<td>
<button class="btn btn-sm btn-success" onclick="smartFixEpisode('${episode.imdb_id}', ${episode.season}, ${episode.episode})">
<button class="btn btn-sm btn-success" onclick="smartFixEpisode('${episode.imdb_id}', ${parseInt(episode.season)}, ${parseInt(episode.episode)})">
<i class="fas fa-magic"></i> Smart Fix
</button>
</td>
@@ -676,6 +677,13 @@ async function smartFixMovie(imdbId) {
}
async function smartFixEpisode(imdbId, season, episode) {
// Validate parameters
if (!imdbId || season === undefined || season === null || episode === undefined || episode === null) {
console.error('smartFixEpisode: Invalid parameters:', {imdbId, season, episode});
showToast('Invalid episode parameters', 'error');
return;
}
try {
const options = await apiCall(`/api/episodes/${imdbId}/${season}/${episode}/date-options`);
showSmartFixModal('episode', options);
@@ -703,7 +711,10 @@ function showSmartFixModal(mediaType, options) {
if (mediaType === 'movie') {
title.textContent = `Fix Date for Movie: ${options.imdb_id}`;
} else {
title.textContent = `Fix Date for Episode: ${options.imdb_id} S${options.season.toString().padStart(2, '0')}E${options.episode.toString().padStart(2, '0')}`;
// Add validation for episode data
const season = options.season || 0;
const episode = options.episode || 0;
title.textContent = `Fix Date for Episode: ${options.imdb_id} S${season.toString().padStart(2, '0')}E${episode.toString().padStart(2, '0')}`;
}
// Build options HTML
@@ -878,7 +889,10 @@ function showEnhancedEditModal(mediaType, options, currentDateadded, currentSour
if (mediaType === 'movie') {
title.textContent = `Edit Movie: ${options.imdb_id}`;
} else {
title.textContent = `Edit Episode: ${options.imdb_id} S${options.season.toString().padStart(2, '0')}E${options.episode.toString().padStart(2, '0')}`;
// Add validation for episode data
const season = options.season || 0;
const episode = options.episode || 0;
title.textContent = `Edit Episode: ${options.imdb_id} S${season.toString().padStart(2, '0')}E${episode.toString().padStart(2, '0')}`;
}
// Build enhanced edit form with date options
@@ -899,7 +913,7 @@ function showEnhancedEditModal(mediaType, options, currentDateadded, currentSour
`;
// Add available date options
options.options.forEach((option, index) => {
(options.date_options || options.options || []).forEach((option, index) => {
const isSelected = option.source === currentSource ? 'checked' : '';
const optionId = `date-option-${index}`;
@@ -1033,25 +1047,39 @@ async function handleEnhancedEditSubmit(event) {
const dateadded = document.getElementById('edit-dateadded').value;
const source = document.getElementById('edit-source').value;
if (!dateadded) {
console.log('🔍 DEBUG: handleEnhancedEditSubmit called with:', {
dateadded: dateadded,
source: source,
mediaType: mediaType,
imdbId: imdbId
});
if (!dateadded && source !== 'airdate') {
showToast('Please enter a date', 'warning');
return;
}
// Convert datetime-local to ISO string with error handling
let isoDateadded = null;
try {
isoDateadded = new Date(dateadded).toISOString();
} catch (e) {
showToast('Invalid date format', 'error');
return;
if (dateadded) {
try {
isoDateadded = new Date(dateadded).toISOString();
} catch (e) {
showToast('Invalid date format', 'error');
return;
}
} else if (source === 'airdate') {
// For airdate source, let the backend handle the date conversion
isoDateadded = null;
}
try {
if (mediaType === 'movie') {
await updateMovieDate(imdbId, isoDateadded, source);
} else {
await updateEpisodeDate(imdbId, options.season, options.episode, isoDateadded, source);
const season = parseInt(document.getElementById('edit-season').value);
const episode = parseInt(document.getElementById('edit-episode').value);
await updateEpisodeDate(imdbId, season, episode, isoDateadded, source);
}
closeModal();
@@ -1062,9 +1090,23 @@ async function handleEnhancedEditSubmit(event) {
}
async function editEpisode(imdbId, season, episode, dateadded, source) {
// Validate parameters
if (!imdbId || season === undefined || season === null || episode === undefined || episode === null) {
console.error('editEpisode: Invalid parameters:', {imdbId, season, episode});
showToast('Invalid episode parameters', 'error');
return;
}
try {
// Load episode options to populate available dates
const options = await apiCall(`/api/episodes/${imdbId}/${season}/${episode}/date-options`);
const dateOptions = await apiCall(`/api/episodes/${imdbId}/${season}/${episode}/date-options`);
// Create options object with episode metadata
const options = {
imdb_id: imdbId,
season: season,
episode: episode,
date_options: dateOptions.options || []
};
showEnhancedEditModal('episode', options, dateadded, source);
} catch (error) {
console.error('Failed to load episode options for edit:', error);
@@ -1244,6 +1286,13 @@ Analysis:
// Episode deletion functionality
async function deleteEpisode(imdbId, season, episode) {
// Validate parameters
if (!imdbId || season === undefined || season === null || episode === undefined || episode === null) {
console.error('deleteEpisode: Invalid parameters:', {imdbId, season, episode});
showToast('Invalid episode parameters', 'error');
return;
}
const episodeStr = `S${season.toString().padStart(2, '0')}E${episode.toString().padStart(2, '0')}`;
// Confirmation dialog
@@ -1252,16 +1301,11 @@ async function deleteEpisode(imdbId, season, episode) {
}
try {
const response = await fetch(`/database/episode/${imdbId}/${season}/${episode}`, {
method: 'DELETE',
headers: {
'Content-Type': 'application/json'
}
const result = await apiCall(`/api/episodes/${imdbId}/${season}/${episode}`, {
method: 'DELETE'
});
const result = await response.json();
if (response.ok && result.success) {
if (result.success) {
showToast(`✅ Episode ${episodeStr} deleted successfully`, 'success');
// Remove the row from the table
@@ -1275,10 +1319,8 @@ async function deleteEpisode(imdbId, season, episode) {
// Update episode counts in modal header
updateEpisodeModalCounts();
} else {
const errorMsg = result.message || result.error || 'Unknown error';
showToast(`❌ Failed to delete episode: ${errorMsg}`, 'error');
showToast(`❌ Failed to delete episode: ${result.message || 'Unknown error'}`, 'error');
}
} catch (error) {
@@ -1295,24 +1337,15 @@ async function deleteMovie(imdbId) {
}
try {
const response = await fetch(`/database/movie/${imdbId}`, {
method: 'DELETE',
headers: {
'Content-Type': 'application/json'
}
const result = await apiCall(`/api/movies/${imdbId}`, {
method: 'DELETE'
});
const result = await response.json();
if (response.ok && result.success) {
if (result.success) {
showToast(`✅ Movie deleted successfully`, 'success');
// Refresh the movies table
loadMovies(currentMoviesPage);
} else {
const errorMsg = result.message || result.error || 'Unknown error';
showToast(`❌ Failed to delete movie: ${errorMsg}`, 'error');
}
} catch (error) {
@@ -1328,20 +1361,19 @@ function updateEpisodeModalCounts() {
const episodesWithDates = Array.from(remainingRows).filter(row =>
row.getAttribute('data-has-date') === 'true'
).length;
const episodesWithVideo = Array.from(remainingRows).filter(row =>
row.getAttribute('data-has-video') === 'true'
).length;
const episodesWithoutDates = totalEpisodes - episodesWithDates;
// Update the stats in the modal
const statsDiv = document.querySelector('.episode-stats');
if (statsDiv) {
// Keep the existing "With Video" count by finding it
const videoCountDiv = statsDiv.querySelector('div:nth-child(4)');
const videoCountText = videoCountDiv ? videoCountDiv.innerHTML : '<div><strong>With Video:</strong> -</div>';
statsDiv.innerHTML = `
<div><strong>Total Episodes:</strong> ${totalEpisodes}</div>
<div><strong>With Dates:</strong> ${episodesWithDates}</div>
<div style="color: #dc3545;"><strong>Missing Dates:</strong> ${episodesWithoutDates}</div>
${videoCountText}
<div><strong>With Video:</strong> ${episodesWithVideo}</div>
`;
}
}
@@ -1656,4 +1688,408 @@ function stopScanStatusPolling() {
clearInterval(scanStatusInterval);
scanStatusInterval = null;
}
}
// ============================================================================
// DATABASE ADMIN TOOLS
// ============================================================================
// Initialize database admin tools
document.addEventListener('DOMContentLoaded', function() {
initializeDatabaseAdminTools();
});
function initializeDatabaseAdminTools() {
// Load quick queries
loadQuickQueries();
// Initialize database query form
const dbQueryForm = document.getElementById('database-query-form');
if (dbQueryForm) {
dbQueryForm.addEventListener('submit', function(e) {
e.preventDefault();
executeDatabaseQuery();
});
}
}
async function loadQuickQueries() {
try {
const response = await apiCall('/api/admin/database/quick-queries');
const select = document.getElementById('quick-query-select');
if (response.success && select) {
// Clear existing options except first
select.innerHTML = '<option value="">Select a predefined query...</option>';
response.queries.forEach(query => {
const option = document.createElement('option');
option.value = query.query;
option.textContent = `${query.name} - ${query.description}`;
select.appendChild(option);
});
}
} catch (error) {
console.error('Failed to load quick queries:', error);
}
}
function loadQuickQuery() {
const select = document.getElementById('quick-query-select');
const textarea = document.getElementById('sql-query');
if (select && textarea && select.value) {
textarea.value = select.value;
}
}
async function executeDatabaseQuery() {
const query = document.getElementById('sql-query').value.trim();
const limit = document.getElementById('query-limit').value;
const resultsDiv = document.getElementById('query-results');
const resultsContent = document.getElementById('query-results-content');
if (!query) {
alert('Please enter a SQL query');
return;
}
try {
resultsContent.innerHTML = '<div class="loading">Executing query...</div>';
resultsDiv.style.display = 'block';
const response = await apiCall('/api/admin/database/query', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ query, limit: parseInt(limit) })
});
if (response.success) {
if (response.columns && response.rows) {
// SELECT query results
let html = `
<div class="query-info">
<strong>Query:</strong> ${response.query}<br>
<strong>Rows returned:</strong> ${response.row_count}
</div>
<div class="table-container">
<table class="results-table">
<thead>
<tr>
`;
response.columns.forEach(col => {
html += `<th>${col}</th>`;
});
html += '</tr></thead><tbody>';
response.rows.forEach(row => {
html += '<tr>';
response.columns.forEach(col => {
const value = row[col];
html += `<td>${value !== null ? value : '<em>NULL</em>'}</td>`;
});
html += '</tr>';
});
html += '</tbody></table></div>';
resultsContent.innerHTML = html;
} else {
// Non-SELECT query
resultsContent.innerHTML = `
<div class="query-info success">
<strong>Query:</strong> ${response.query}<br>
<strong>Result:</strong> ${response.message}<br>
<strong>Rows affected:</strong> ${response.rows_affected}
</div>
`;
}
} else {
resultsContent.innerHTML = `
<div class="query-info error">
<strong>Error:</strong> ${response.error}<br>
<strong>Query:</strong> ${response.query}
</div>
`;
}
} catch (error) {
resultsContent.innerHTML = `
<div class="query-info error">
<strong>Error:</strong> Failed to execute query: ${error.message}
</div>
`;
}
}
// NFO File Repair Functions
async function checkMissingNFODates() {
const resultsDiv = document.getElementById('nfo-scan-results');
const resultsContent = document.getElementById('nfo-scan-content');
try {
resultsContent.innerHTML = '<div class="loading">Scanning NFO files for missing dateadded elements...</div>';
resultsDiv.style.display = 'block';
const response = await apiCall('/api/admin/nfo/missing-dateadded');
if (response.success) {
let html = `
<div class="scan-summary">
<h4>NFO Scan Results</h4>
<p><strong>Total episodes checked:</strong> ${response.total_episodes_checked || 0}</p>
<p><strong>Total movies checked:</strong> ${response.total_movies_checked || 0}</p>
<p><strong>Total items checked:</strong> ${response.total_items_checked || 0}</p>
<p><strong>Items missing dateadded in NFO:</strong> ${response.missing_dateadded_count}</p>
</div>
`;
if (response.missing_dateadded_count > 0) {
html += `
<div class="missing-episodes">
<h5>Items with missing dateadded (showing first 50):</h5>
<div class="nfo-repair-table-container">
<table class="nfo-repair-table">
<thead>
<tr>
<th style="width: 8%">Type</th>
<th style="width: 12%">IMDb ID</th>
<th style="width: 8%">Season</th>
<th style="width: 8%">Episode</th>
<th style="width: 15%">Database Date</th>
<th style="width: 10%">Source</th>
<th style="width: 25%">Series/Movie</th>
<th style="width: 14%">Actions</th>
</tr>
</thead>
<tbody>
`;
(response.items || response.episodes || []).forEach(item => {
const displayDate = item.dateadded ? new Date(item.dateadded).toLocaleString() : 'None';
const seriesName = item.series_name || item.title || 'Unknown';
const nfoFileName = item.nfo_path ? item.nfo_path.split('/').pop() : 'Unknown';
html += `
<tr>
<td><span class="media-type-badge ${item.media_type || 'episode'}">${item.media_type || 'episode'}</span></td>
<td><code>${item.imdb_id}</code></td>
<td class="text-center">${item.season || 'N/A'}</td>
<td class="text-center">${item.episode || 'N/A'}</td>
<td><small>${displayDate}</small></td>
<td><small>${item.source || 'unknown'}</small></td>
<td><strong>${seriesName}</strong><br><small class="text-muted">${nfoFileName}</small></td>
<td>
${item.dateadded ?
'<span class="status-ready">✅ Ready to fix</span>' :
'<span class="status-missing">❌ No DB date</span>'
}
</td>
</tr>
`;
});
html += `
</tbody>
</table>
</div>
<div class="nfo-fix-actions">
<button onclick="fixAllMissingNFODates()" class="btn btn-primary">
🔧 Fix All Missing dateadded Elements
</button>
<p class="text-muted">This will automatically add dateadded elements to NFO files using database values.</p>
</div>
</div>`;
} else {
html += '<div class="success">✅ All NFO files have dateadded elements!</div>';
}
resultsContent.innerHTML = html;
} else {
resultsContent.innerHTML = `<div class="error">Error: ${response.message}</div>`;
}
} catch (error) {
resultsContent.innerHTML = `<div class="error">Failed to scan NFO files: ${error.message}</div>`;
}
}
async function fixAllMissingNFODates() {
if (!confirm('This will update all NFO files that are missing dateadded elements. Continue?')) {
return;
}
const resultsContent = document.getElementById('nfo-scan-content');
try {
resultsContent.innerHTML = '<div class="loading">Fixing all missing dateadded elements in NFO files...</div>';
const response = await apiCall('/api/admin/nfo/bulk-update', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ fix_all: true })
});
if (response.success) {
let html = `
<div class="fix-summary success">
<h4>NFO Fix Results</h4>
<p>✅ <strong>${response.updated_count}</strong> NFO files updated successfully</p>
<p><strong>Total episodes processed:</strong> ${response.total_episodes}</p>
<p><strong>Message:</strong> ${response.message}</p>
</div>
`;
if (response.errors && response.errors.length > 0) {
html += `
<div class="fix-errors">
<h5>Errors (first 10):</h5>
<ul>
`;
response.errors.forEach(error => {
html += `<li>${error}</li>`;
});
html += '</ul></div>';
}
resultsContent.innerHTML = html;
} else {
resultsContent.innerHTML = `<div class="error">Error: ${response.message}</div>`;
}
} catch (error) {
resultsContent.innerHTML = `<div class="error">Failed to fix NFO files: ${error.message}</div>`;
}
}
async function fixSpecificSeries() {
const imdbIds = prompt('Enter IMDb IDs to fix (comma-separated):\nExample: tt9111220,tt12327578');
if (!imdbIds) return;
const idArray = imdbIds.split(',').map(id => id.trim()).filter(id => id);
if (idArray.length === 0) {
alert('Please enter valid IMDb IDs');
return;
}
const resultsContent = document.getElementById('nfo-scan-content');
try {
resultsContent.innerHTML = '<div class="loading">Fixing NFO files for specified series...</div>';
const response = await apiCall('/api/admin/nfo/bulk-update', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ imdb_ids: idArray })
});
if (response.success) {
let html = `
<div class="fix-summary success">
<h4>NFO Fix Results for Specific Series</h4>
<p>✅ <strong>${response.updated_count}</strong> NFO files updated successfully</p>
<p><strong>Series processed:</strong> ${idArray.join(', ')}</p>
<p><strong>Message:</strong> ${response.message}</p>
</div>
`;
resultsContent.innerHTML = html;
} else {
resultsContent.innerHTML = `<div class="error">Error: ${response.message}</div>`;
}
} catch (error) {
resultsContent.innerHTML = `<div class="error">Failed to fix NFO files: ${error.message}</div>`;
}
}
// Database Admin Functions
async function showEpisodesMissingNFODateadded() {
const resultsDiv = document.getElementById('admin-results');
const resultsContent = document.getElementById('admin-results-content');
try {
resultsContent.innerHTML = '<div class="loading">Checking for episodes missing NFO dateadded...</div>';
resultsDiv.style.display = 'block';
const response = await apiCall('/api/admin/nfo/missing-dateadded');
if (response.success) {
const html = `
<h4>Episodes Missing NFO dateadded</h4>
<p><strong>Found:</strong> ${response.missing_dateadded_count} episodes missing dateadded in NFO files</p>
<p><strong>Total checked:</strong> ${response.total_episodes_checked} episodes</p>
${response.missing_dateadded_count > 0 ?
'<p><strong>Recommendation:</strong> Use the "NFO File Repair" tool to fix these issues.</p>' :
'<p>✅ All NFO files are properly maintained!</p>'
}
`;
resultsContent.innerHTML = html;
} else {
resultsContent.innerHTML = `<div class="error">Error: ${response.message}</div>`;
}
} catch (error) {
resultsContent.innerHTML = `<div class="error">Failed to check NFO files: ${error.message}</div>`;
}
}
async function exportDatabaseReport() {
alert('Database export functionality will be implemented in a future update.');
}
async function showRecentWebhookActivity() {
const resultsDiv = document.getElementById('admin-results');
const resultsContent = document.getElementById('admin-results-content');
try {
resultsContent.innerHTML = '<div class="loading">Loading recent webhook activity...</div>';
resultsDiv.style.display = 'block';
const response = await apiCall('/api/admin/database/query', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
query: "SELECT * FROM episodes WHERE last_updated > NOW() - INTERVAL '24 hours' ORDER BY last_updated DESC",
limit: 20
})
});
if (response.success && response.rows) {
let html = `
<h4>Recent Webhook Activity (Last 24 Hours)</h4>
<p><strong>Episodes processed:</strong> ${response.row_count}</p>
<div class="table-container">
<table class="results-table">
<thead>
<tr>
<th>IMDb ID</th>
<th>Season</th>
<th>Episode</th>
<th>Date Added</th>
<th>Source</th>
<th>Last Updated</th>
</tr>
</thead>
<tbody>
`;
response.rows.forEach(row => {
html += `
<tr>
<td>${row.imdb_id}</td>
<td>${row.season}</td>
<td>${row.episode}</td>
<td>${row.dateadded ? new Date(row.dateadded).toLocaleString() : 'None'}</td>
<td>${row.source}</td>
<td>${new Date(row.last_updated).toLocaleString()}</td>
</tr>
`;
});
html += '</tbody></table></div>';
resultsContent.innerHTML = html;
} else {
resultsContent.innerHTML = '<p>No recent webhook activity found.</p>';
}
} catch (error) {
resultsContent.innerHTML = `<div class="error">Failed to load webhook activity: ${error.message}</div>`;
}
}
+54 -26
View File
@@ -797,7 +797,7 @@ class TVProcessor:
# Get episode date information - webhook processing prioritizes existing DB entries
_log("DEBUG", f"Processing webhook episode: IMDb={imdb_id}, S{season_num:02d}E{episode_num:02d}")
aired, dateadded, source = self._get_webhook_episode_date(imdb_id, season_num, episode_num, series_metadata)
aired, dateadded, source = self._get_webhook_episode_date(imdb_id, season_num, episode_num, series_metadata, webhook_episode)
enhanced_metadata = self._get_episode_metadata(series_metadata, season_num, episode_num, season_dir) if series_metadata else self._get_episode_metadata(None, season_num, episode_num, season_dir)
# Create NFO
@@ -825,20 +825,20 @@ class TVProcessor:
episodes_processed += 1
# Create season/tvshow NFOs if any episodes were processed
if episodes_processed > 0 and config.manage_nfo:
seasons_processed = set()
for webhook_episode in webhook_episodes:
season_num = webhook_episode.get("seasonNumber")
if season_num and season_num not in seasons_processed:
season_dir = series_path / config.tv_season_dir_format.format(season=season_num)
if season_dir.exists():
self.nfo_manager.create_season_nfo(season_dir, season_num)
seasons_processed.add(season_num)
# Get TVDB ID for better Emby compatibility
tvdb_id = self.external_clients.get_tvdb_series_id(imdb_id)
self.nfo_manager.create_tvshow_nfo(series_path, imdb_id, tvdb_id)
# DISABLED: Skip creating season.nfo and tvshow.nfo files (user only wants episode NFOs)
# if episodes_processed > 0 and config.manage_nfo:
# seasons_processed = set()
# for webhook_episode in webhook_episodes:
# season_num = webhook_episode.get("seasonNumber")
# if season_num and season_num not in seasons_processed:
# season_dir = series_path / config.tv_season_dir_format.format(season=season_num)
# if season_dir.exists():
# self.nfo_manager.create_season_nfo(season_dir, season_num)
# seasons_processed.add(season_num)
#
# # Get TVDB ID for better Emby compatibility
# tvdb_id = self.external_clients.get_tvdb_series_id(imdb_id)
# self.nfo_manager.create_tvshow_nfo(series_path, imdb_id, tvdb_id)
_log("INFO", f"Completed targeted processing: {episodes_processed}/{len(webhook_episodes)} episodes processed")
@@ -942,17 +942,46 @@ class TVProcessor:
return None
def _get_webhook_episode_date(self, imdb_id: str, season_num: int, episode_num: int, series_metadata: Optional[Dict[str, Any]] = None) -> Tuple[Optional[str], Optional[str], str]:
def _get_webhook_episode_date(self, imdb_id: str, season_num: int, episode_num: int, series_metadata: Optional[Dict[str, Any]] = None, webhook_episode: Optional[Dict[str, Any]] = None) -> Tuple[Optional[str], Optional[str], str]:
"""
Get episode date for webhook processing - avoid treating webhook as gospel.
Get episode date for webhook processing - database-first approach.
Logic:
1. Check Sonarr import history FIRST (any history = show has existed for a while)
2. If ANY import history exists Use air date (webhook is likely upgrade/rename)
3. If NO import history Use webhook date (truly new show)
1. Check NFOGuard database FIRST - if we have a date, use it (no re-processing)
2. No date in DB? Check Sonarr import history with our rules:
- Import before rename = use import date
- Rename before import = use airdate
- First import matches webhook = valid new episode
3. Final fallback = airdate
This prevents upgrades from overriding dates for shows you've had for months/years.
This prevents webhooks from overriding existing good data.
"""
# STEP 1: Check NFOGuard database first
existing_entry = None
existing_date = None
try:
existing_entry = self.db.get_episode(imdb_id, season_num, episode_num)
if existing_entry and existing_entry.get('date_added'):
existing_date = existing_entry['date_added']
_log("INFO", f"Found existing date in database for S{season_num:02d}E{episode_num:02d}: {existing_date}")
# For webhook processing, use existing data (fast path)
# Manual scans will validate below
# Still get aired date for NFO
aired = None
if series_metadata and 'episodes' in series_metadata:
episode_data = series_metadata['episodes'].get((season_num, episode_num))
if episode_data:
aired = episode_data.get('airDate')
return aired, str(existing_date), "database:existing"
except Exception as e:
_log("DEBUG", f"Database check failed for S{season_num:02d}E{episode_num:02d}: {e}")
_log("INFO", f"No existing date in database for S{season_num:02d}E{episode_num:02d}, checking Sonarr import history")
# Get aired date and episode ID from Sonarr
aired = None
episode_id = None
@@ -963,8 +992,8 @@ class TVProcessor:
episode_id = episode_data.get('id')
_log("DEBUG", f"Got aired date from Sonarr for S{season_num:02d}E{episode_num:02d}: {aired}")
# STEP 1: Check Sonarr import history FIRST (this is the key check)
_log("INFO", f"Checking Sonarr import history for S{season_num:02d}E{episode_num:02d} to detect import vs rename events")
# STEP 2: Check Sonarr import history (authoritative source)
_log("INFO", f"Checking Sonarr import history for S{season_num:02d}E{episode_num:02d}")
if episode_id and hasattr(self, 'sonarr'):
try:
@@ -973,9 +1002,8 @@ class TVProcessor:
_log("DEBUG", f"Import history result: {import_history}")
if import_history:
# Found actual import event - use this date
_log("INFO", f"Found real import event for S{season_num:02d}E{episode_num:02d}: {import_history}")
_log("INFO", f"Using first import date (not webhook): {import_history}")
# Found actual import event from Sonarr
_log("INFO", f"Found Sonarr import event for S{season_num:02d}E{episode_num:02d}: {import_history}")
return aired, import_history, "sonarr:import_history"
else:
# No import events found - this means only renames/moves exist in history
+19 -18
View File
@@ -371,21 +371,22 @@ class TVSeriesProcessor:
return None, "no_external_data"
def _create_series_nfos(self, series_path: Path, imdb_id: str):
"""Create tvshow.nfo and season.nfo files only if they don't exist"""
# Create tvshow.nfo only if it doesn't exist
tvshow_nfo = series_path / "tvshow.nfo"
if not tvshow_nfo.exists():
self.nfo_manager.create_tvshow_nfo(series_path, imdb_id)
else:
_log("DEBUG", f"Skipping tvshow.nfo creation - already exists: {tvshow_nfo}")
# Create season.nfo for each season directory only if they don't exist
for season_dir in series_path.iterdir():
if season_dir.is_dir() and self._is_season_directory(season_dir.name):
season_num = self._extract_season_number(season_dir.name)
if season_num is not None:
season_nfo = season_dir / "season.nfo"
if not season_nfo.exists():
self.nfo_manager.create_season_nfo(season_dir, season_num)
else:
_log("DEBUG", f"Skipping season.nfo creation - already exists: {season_nfo}")
"""DISABLED: Skip creating tvshow.nfo and season.nfo files (user only wants episode NFOs)"""
# # Create tvshow.nfo only if it doesn't exist
# tvshow_nfo = series_path / "tvshow.nfo"
# if not tvshow_nfo.exists():
# self.nfo_manager.create_tvshow_nfo(series_path, imdb_id)
# else:
# _log("DEBUG", f"Skipping tvshow.nfo creation - already exists: {tvshow_nfo}")
#
# # Create season.nfo for each season directory only if they don't exist
# for season_dir in series_path.iterdir():
# if season_dir.is_dir() and self._is_season_directory(season_dir.name):
# season_num = self._extract_season_number(season_dir.name)
# if season_num is not None:
# season_nfo = season_dir / "season.nfo"
# if not season_nfo.exists():
# self.nfo_manager.create_season_nfo(season_dir, season_num)
# else:
# _log("DEBUG", f"Skipping season.nfo creation - already exists: {season_nfo}")
pass # Function disabled - only process episode NFOs
+23 -6
View File
@@ -5,18 +5,19 @@ Simple script to start web interface using existing config system
"""
import os
import sys
import time
import uvicorn
from fastapi import FastAPI
from fastapi.staticfiles import StaticFiles
from fastapi.responses import FileResponse
# Import existing configuration
# Import existing configuration (keep using core config for simplicity)
from config.settings import config
# Import existing database and components
# Import existing database and components
from core.database import NFOGuardDatabase
# Import web routes from existing system
# Import web routes from existing system (now includes DELETE route)
from api.web_routes import register_web_routes
# Import authentication system
@@ -82,6 +83,22 @@ def setup_static_files(app: FastAPI) -> None:
# Return 204 No Content if no favicon found
from fastapi import Response
return Response(status_code=204)
# Health check endpoint for Docker
@app.get("/health")
async def health_check():
"""Health check endpoint for Docker container monitoring"""
try:
# Basic health check - verify the web service is responsive
return {
"status": "healthy",
"service": "nfoguard-web",
"timestamp": time.time(),
"version": "2.8.0-web"
}
except Exception as e:
from fastapi import HTTPException
raise HTTPException(status_code=503, detail=f"Health check failed: {e}")
def main():
@@ -116,7 +133,7 @@ def main():
else:
print("🌐 Web authentication disabled")
# Create dependencies for dependency injection (simplified for web-only)
# Create dependencies for dependency injection
dependencies = {
"db": db,
"config": config,
@@ -129,15 +146,15 @@ def main():
# Add authentication middleware if enabled (BEFORE routes)
if auth_enabled:
# Pass the session manager to middleware so it uses the same instance
app.add_middleware(SimpleAuthMiddleware, config=config, session_manager=session_manager)
print("🔐 Authentication middleware added to web interface")
# Setup static files and routes
setup_static_files(app)
# Register web routes
# Register web routes (now includes DELETE /api/episodes/ route)
register_web_routes(app, dependencies)
print("✅ Registered web routes with DELETE /api/episodes/ support")
print(f"🚀 Starting web server on {web_host}:{web_port}")
+31 -4
View File
@@ -667,6 +667,13 @@ async function smartFixMovie(imdbId) {
}
async function smartFixEpisode(imdbId, season, episode) {
// Validate parameters
if (!imdbId || season === undefined || season === null || episode === undefined || episode === null) {
console.error('smartFixEpisode: Invalid parameters:', {imdbId, season, episode});
showToast('Invalid episode parameters', 'error');
return;
}
try {
const options = await apiCall(`/api/episodes/${imdbId}/${season}/${episode}/date-options`);
showSmartFixModal('episode', options);
@@ -694,7 +701,10 @@ function showSmartFixModal(mediaType, options) {
if (mediaType === 'movie') {
title.textContent = `Fix Date for Movie: ${options.imdb_id}`;
} else {
title.textContent = `Fix Date for Episode: ${options.imdb_id} S${options.season.toString().padStart(2, '0')}E${options.episode.toString().padStart(2, '0')}`;
// Add validation for episode data
const season = options.season || 0;
const episode = options.episode || 0;
title.textContent = `Fix Date for Episode: ${options.imdb_id} S${season.toString().padStart(2, '0')}E${episode.toString().padStart(2, '0')}`;
}
// Build options HTML
@@ -910,7 +920,10 @@ function showEnhancedEditModal(mediaType, options, currentDateadded, currentSour
if (mediaType === 'movie') {
title.textContent = `Edit Movie: ${options.imdb_id}`;
} else {
title.textContent = `Edit Episode: ${options.imdb_id} S${options.season.toString().padStart(2, '0')}E${options.episode.toString().padStart(2, '0')}`;
// Add validation for episode data
const season = options.season || 0;
const episode = options.episode || 0;
title.textContent = `Edit Episode: ${options.imdb_id} S${season.toString().padStart(2, '0')}E${episode.toString().padStart(2, '0')}`;
}
// Build enhanced edit form with date options
@@ -1094,6 +1107,13 @@ async function handleEnhancedEditSubmit(event) {
}
async function editEpisode(imdbId, season, episode, dateadded, source) {
// Validate parameters
if (!imdbId || season === undefined || season === null || episode === undefined || episode === null) {
console.error('editEpisode: Invalid parameters:', {imdbId, season, episode});
showToast('Invalid episode parameters', 'error');
return;
}
try {
// Load episode options to populate available dates
const options = await apiCall(`/api/episodes/${imdbId}/${season}/${episode}/date-options`);
@@ -1276,6 +1296,13 @@ Analysis:
// Episode deletion functionality
async function deleteEpisode(imdbId, season, episode) {
// Validate parameters
if (!imdbId || season === undefined || season === null || episode === undefined || episode === null) {
console.error('deleteEpisode: Invalid parameters:', {imdbId, season, episode});
showToast('Invalid episode parameters', 'error');
return;
}
const episodeStr = `S${season.toString().padStart(2, '0')}E${episode.toString().padStart(2, '0')}`;
// Confirmation dialog
@@ -1284,7 +1311,7 @@ async function deleteEpisode(imdbId, season, episode) {
}
try {
const response = await fetch(`/database/episode/${imdbId}/${season}/${episode}`, {
const response = await fetch(`/api/episodes/${imdbId}/${season}/${episode}`, {
method: 'DELETE',
headers: {
'Content-Type': 'application/json'
@@ -1327,7 +1354,7 @@ async function deleteMovie(imdbId) {
}
try {
const response = await fetch(`/database/movie/${imdbId}`, {
const response = await fetch(`/api/movies/${imdbId}`, {
method: 'DELETE',
headers: {
'Content-Type': 'application/json'
+31 -1
View File
@@ -3,6 +3,7 @@ Webhook Batching System for NFOGuard
Handles batching and processing of webhook events to avoid processing storms
"""
import threading
import time
from pathlib import Path
from typing import Dict, Set
from concurrent.futures import ThreadPoolExecutor
@@ -144,7 +145,13 @@ class WebhookBatcher:
if processing_mode == 'targeted' and episodes_data:
_log("INFO", f"Using targeted episode processing for {len(episodes_data)} episodes")
self.tv_processor.process_webhook_episodes(path_obj, episodes_data)
# Handle sequential processing for bulk downloads
if len(episodes_data) > 1 and config.sequential_delay > 0:
_log("INFO", f"Processing {len(episodes_data)} episodes sequentially with {config.sequential_delay}s delay")
self._process_episodes_sequentially(path_obj, episodes_data)
else:
self.tv_processor.process_webhook_episodes(path_obj, episodes_data)
else:
_log("INFO", f"Using series processing mode (fallback or configured)")
self.tv_processor.process_series(path_obj)
@@ -164,6 +171,29 @@ class WebhookBatcher:
with self.lock:
self.processing.discard(key)
def _process_episodes_sequentially(self, path_obj: Path, episodes_data: list):
"""Process episodes one by one with delays to avoid API spam"""
total_episodes = len(episodes_data)
for i, episode in enumerate(episodes_data, 1):
try:
season = episode.get('seasonNumber', '?')
episode_num = episode.get('episodeNumber', '?')
_log("INFO", f"Processing episode {i}/{total_episodes}: S{season:02d}E{episode_num:02d}")
# Process single episode
self.tv_processor.process_webhook_episodes(path_obj, [episode])
# Add delay between episodes (except for the last one)
if i < total_episodes and config.sequential_delay > 0:
_log("INFO", f"Waiting {config.sequential_delay}s before next episode...")
time.sleep(config.sequential_delay)
except Exception as e:
_log("ERROR", f"Error processing episode {i}/{total_episodes}: {e}")
# Continue with next episode even if one fails
_log("INFO", f"Completed sequential processing of {total_episodes} episodes")
def get_status(self) -> Dict:
"""Get batch queue status"""
with self.lock: