sbcrumb/nfoguard
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement optimal caching hierarchy for both movies and TV shows
Local Docker Build (Dev) / build-dev (pull_request) Successful in 4s
Details

Browse Source 
Perfect the caching strategy as requested:
1. Database check first (fastest lookup)
2. NFO file extraction with database caching
3. API calls only when absolutely necessary

Movies: Fixed order from NFO->DB->API to DB->NFO->API
TV Shows: Added missing NFO extraction tier between database and API

Benefits:
- First scan: Extract existing NFOGuard data from NFOs and cache in DB
- Subsequent scans: ~99% database cache hits, minimal API calls
- NFO recovery: Rebuild database from existing NFO files
- Webhook processing: Cache webhook dates in database immediately

Expected performance: Manual scans now use 3-tier optimization
This commit is contained in:
sbcrumb
2025-10-10 15:47:53 -04:00
parent 88b75230a7
commit 7e56ac1601
3 changed files with 93 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files
processors/tv_processor.py
+48 -7
View File
@@ -106,9 +106,10 @@ class TVProcessor:
episode_dates = {}
episodes_needing_lookup = []
# OPTIMIZATION: Check database first for existing dates
_log("DEBUG", f"Checking database for existing episode dates for {len(disk_episodes)} episodes")
# TIER 1: Check database first for existing dates (fastest)
_log("DEBUG", f"TIER 1 - Checking database for existing episode dates for {len(disk_episodes)} episodes")
db_cache_hits = 0
episodes_needing_nfo_check = []
for (season, episode) in disk_episodes:
# Try database first - this is much faster than API calls
@@ -123,14 +124,54 @@ class TVProcessor:
db_cache_hits += 1
_log("DEBUG", f"Database cache hit for S{season:02d}E{episode:02d}: {dateadded}")
else:
# Not in database or incomplete - needs lookup
episodes_needing_lookup.append((season, episode))
# Not in database or incomplete - needs NFO check
episodes_needing_nfo_check.append((season, episode))
_log("INFO", f"Database cache hits: {db_cache_hits}/{len(disk_episodes)} episodes. Need API lookup: {len(episodes_needing_lookup)}")
_log("INFO", f"Database cache hits: {db_cache_hits}/{len(disk_episodes)} episodes. Need NFO check: {len(episodes_needing_nfo_check)}")
# Only call Sonarr API for episodes not in database
# TIER 2: Check NFO files for NFOGuard dates and cache them in database
nfo_cache_hits = 0
episodes_needing_lookup = []
if episodes_needing_nfo_check:
_log("DEBUG", f"TIER 2 - Checking NFO files for NFOGuard dates for {len(episodes_needing_nfo_check)} episodes")
for (season, episode) in episodes_needing_nfo_check:
# Look for existing NFO files for this episode
season_dir = series_path / config.tv_season_dir_format.format(season=season)
episode_files = disk_episodes[(season, episode)]
nfo_found = False
for episode_file in episode_files:
# Try to find matching NFO file
nfo_path = episode_file.with_suffix('.nfo')
if nfo_path.exists():
# Extract NFOGuard data from episode NFO
nfo_data = self.nfo_manager.extract_nfoguard_dates_from_nfo(nfo_path)
if nfo_data:
aired = nfo_data.get('aired')
dateadded = nfo_data.get('dateadded')
source = nfo_data.get('source', 'nfo_cache')
if dateadded:
episode_dates[(season, episode)] = (aired, dateadded, source)
nfo_cache_hits += 1
nfo_found = True
# Cache NFO data in database for future lookups
self.db.upsert_episode_date(imdb_id, season, episode, aired, dateadded, source, True)
_log("DEBUG", f"NFO cache hit for S{season:02d}E{episode:02d}: {dateadded} - cached in DB")
break
if not nfo_found:
# No NFO data found - needs API lookup
episodes_needing_lookup.append((season, episode))
_log("INFO", f"NFO cache hits: {nfo_cache_hits}/{len(episodes_needing_nfo_check)} episodes. Need API lookup: {len(episodes_needing_lookup)}")
# TIER 3: Only call Sonarr API for episodes not in database or NFO files
if episodes_needing_lookup:
_log("DEBUG", f"Querying Sonarr for {len(episodes_needing_lookup)} episodes missing from database")
_log("DEBUG", f"TIER 3 - Querying Sonarr for {len(episodes_needing_lookup)} episodes missing from database and NFO files")
sonarr_episodes = self._get_sonarr_episodes(imdb_id, episodes_needing_lookup)
# Process episodes that needed lookup