Merge remote-tracking branch 'origin/HOME' into HOME

.specstory/history/2025-11-10_16-59Z-change-compression-method-to-7z.md

@@ -19562,7 +19562,6 @@ _**Agent (model default, mode Agent)**_
 
 ---
 
-
 Updating the logic to cap workers at the number of sequences to zip. Checking how `max_workers` is called:
 
 
@@ -19599,8 +19598,6 @@ Output mode: content
 
 
 
-
-
 ---
 
 
@@ -19617,15 +19614,12 @@ Output mode: content
 
 
 
-
-
 ---
 
 
 
 ---
 
-
 Updating `max_workers` to cap workers at the number of work items:
 
 
@@ -19643,7 +19637,6 @@ Updating `max_workers` to cap workers at the number of work items:
 
 ---
 
-
 Capping workers at the number of work items:
 
 
@@ -19692,7 +19685,6 @@ Lines added: 1, lines removed: 1
 
 ---
 
-
 Updating the configured `MAX_7Z_INSTANCES` case to also cap at the number of work items:
 
 
@@ -19742,7 +19734,6 @@ Lines added: 1, lines removed: 1
 
 ---
 
-
 Updating the default case when RAM detection fails:
 
 
@@ -19809,15 +19800,12 @@ Lint results
 
 
 
-
-
 ---
 
 
 
 ---
 
-
 Updated the code to cap worker count at the number of sequences to zip. Changes:
 
 1. **Auto-calculate mode (`Max7zInst=0`)**: Caps workers at the number of work items
@@ -19832,3 +19820,828 @@ The script still uses aggressive RAM allocation when there are many sequences, b
 
 ---
 
+_**User (2025-11-12 16:48Z)**_
+
+pulled from HOME again.@2025-11-12_01-02Z-compression-failure-and-memory-allocation-issues.md  the RAM estimate is far higher than the actual result.
+
+
+
+PS P:\250827_FestivalTurf\Renders> & P:\250827_FestivalTurf\Renders\ZipSeqArchv.bat
+
+
+
+[init] Loading configuration sources...
+
+[init] Reading project config at P:\250827_FestivalTurf\.config\config.json
+
+[init] Configuration loaded from project
+
+[init] zip_sequences starting up...
+
+[init] Working directory: P:\250827_FestivalTurf
+
+[init] Mode: zip, zipper: 7z, jobs arg: auto
+
+[init] 7z executable: C:\ProgramData\chocolatey\bin\7z.EXE, Max7zInst: auto
+
+[init] Scanning sequences under \\NEXUS\proj\250827_FestivalTurf\Renders
+
+[scan] [1] Inspecting 01 Develop Your Turf Plan\01_Blueprint
+
+[scan] 01 Develop Your Turf Plan\01_Blueprint metadata unchanged; archive up to date
+
+[scan] [2] Inspecting 01 Develop Your Turf Plan\01_Blueprint_insert
+
+[scan] 01 Develop Your Turf Plan\01_Blueprint_insert metadata unchanged; archive up to date
+
+[scan] [3] Inspecting 01 Develop Your Turf Plan\02_Roll Sizes
+
+[scan] 01 Develop Your Turf Plan\02_Roll Sizes metadata unchanged; archive up to date
+
+[scan] [4] Inspecting 01 Develop Your Turf Plan\03_Window
+
+[scan] 01 Develop Your Turf Plan\03_Window metadata unchanged; archive up to date
+
+[scan] [5] Inspecting 01 Develop Your Turf Plan\04_Dog
+
+[scan] 01 Develop Your Turf Plan\04_Dog metadata unchanged; archive up to date
+
+[scan] [10] Inspecting 02 Preparing the Base\01_intro
+
+[scan] [20] Inspecting 02 Preparing the Base\10_FT_Talking
+
+[scan] [30] Inspecting 03 Roll Out And Stretch\05_diagram
+
+[scan] Skipping 04 Securing Your Seam\02_FT talking (unchanged since archive)
+
+[scan] Skipping 04 Securing Your Seam\02_FT talking_Pt2 (unchanged since archive)
+
+[scan] [40] Inspecting 04 Securing Your Seam\02_S seam insert
+
+[scan] Skipping 04 Securing Your Seam\04_holding nails (unchanged since archive)
+
+[scan] [50] Inspecting 04 Securing Your Seam\Nail_dimpling_example
+
+[scan] Queued 05 Stretch Cut Nail\01_intro for compression (~1.10GB) [1 total]
+
+[scan] Skipping 05 Stretch Cut Nail\02_kicker insert (unchanged since archive)
+
+[scan] Queued 05 Stretch Cut Nail\02_talking for compression (~0.81GB) [2 total]
+
+[scan] Queued 05 Stretch Cut Nail\03_FT Shuffle for compression (~0.70GB) [3 total]
+
+[scan] Queued 05 Stretch Cut Nail\03_FT Shuffle_dance_only for compression (~0.38GB) [4 total]
+
+[scan] Queued 05 Stretch Cut Nail\03_FT Shuffle_Intro for compression (~0.46GB) [5 total]
+
+[scan] [60] Inspecting 05 Stretch Cut Nail\Visual 10
+
+[scan] Queued 05 Stretch Cut Nail\Visual 11 for compression (~1.17GB) [10 total]
+
+[scan] Queued 05 Stretch Cut Nail\Visual 9 for compression (~0.37GB) [15 total]
+
+[zip] RAM: 102.2GB available (80% of total), 97.1GB for compression (95%)
+
+[zip] Estimated RAM per sequence: 181.6GB total across 15 sequences
+
+[zip] Bin-packing: 2 workers, bin sizes: ['97.1GB', '84.5GB']
+
+[zip] Final worker count: 2 (requested: None)
+
+[init] Preparing to compress 15 sequence(s) with 2 worker(s)
+
+[zip] Renders\05 Stretch Cut Nail\01_intro -> Renders\_zipped\05 Stretch Cut Nail\01_intro.7z
+
+[zip] Renders\05 Stretch Cut Nail\02_talking -> Renders\_zipped\05 Stretch Cut Nail\02_talking.7z
+
+[zip] Renders\05 Stretch Cut Nail\03_FT Shuffle -> Renders\_zipped\05 Stretch Cut Nail\03_FT Shuffle.7z
+
+[zip] 1/15 05 Stretch Cut Nail\02_talking
+
+---
+
+_**Agent (model default, mode Agent)**_
+
+
+
+---
+
+
+The RAM estimate is too high. The image shows 7z processes using ~7–10GB each, but the estimate is ~12GB per sequence. Updating the estimation to match actual usage:
+
+
+---
+
+<tool-use data-tool-type="read" data-tool-name="read_file">
+<details><summary>Tool use: **read_file** • Read file: zip_sequences.py</summary>
+
+</details>
+</tool-use>
+
+---
+
+
+
+---
+
+<tool-use data-tool-type="read" data-tool-name="read_file">
+<details><summary>Tool use: **read_file** • Read file: zip_sequences.py</summary>
+
+</details>
+</tool-use>
+
+---
+
+
+
+
+
+---
+
+
+
+---
+
+
+Updating the RAM estimate to match actual usage (~7–10GB per 7z process). The current estimate is too high:
+
+
+---
+
+<tool-use data-tool-type="write" data-tool-name="search_replace">
+
+<details><summary>Tool use: **code_edit** • Edit file: zip_sequences.py</summary>
+
+**Chunk 1**
+Lines added: 12, lines removed: 17
+
+```diff
+@@ -214,23 +214,18 @@
+          FIXED_DICT_SIZE_BYTES = FIXED_DICT_SIZE_MB * 1024 * 1024
+          
+-         # Dictionary RAM: ~11x dictionary size (1GB dict = 11GB RAM for dictionary operations)
+-         # This is the main memory consumer for 7z LZMA compression
+-         dict_ram = FIXED_DICT_SIZE_BYTES * 11  # 11GB for 1GB dictionary
++         # Dictionary RAM: ~8x dictionary size (1GB dict = 8GB RAM)
++         # Based on actual observed usage: 7z processes use ~7-10GB with 1GB dictionary
++         dict_ram = FIXED_DICT_SIZE_BYTES * 8  # 8GB for 1GB dictionary
+          
+-         # Input buffer: 7z processes in chunks, but for very large sequences needs more RAM
+-         # For sequences &gt;100GB, use a larger buffer factor to handle file metadata and processing
+-         if total_bytes &gt; 500 * 1024 * 1024 * 1024:  # &gt;500GB (extremely large)
+-             # Extremely large sequences: use 8% of size, capped at 64GB
+-             # This accounts for file metadata, directory structures, and processing overhead
+-             input_buffer = min(int(total_bytes * 0.08), 64 * 1024 * 1024 * 1024)
+-             overhead = 8 * 1024 * 1024 * 1024  # 8GB overhead for extremely large sequences
+-         elif total_bytes &gt; 100 * 1024 * 1024 * 1024:  # &gt;100GB
+-             # Very large sequences: use 5% of size, capped at 32GB
+-             input_buffer = min(int(total_bytes * 0.05), 32 * 1024 * 1024 * 1024)
+-             overhead = 4 * 1024 * 1024 * 1024  # 4GB for large sequences
++         # Input buffer: 7z processes files in chunks, minimal buffer needed
++         # For most sequences, a small fixed buffer is sufficient
++         if total_bytes &gt; 100 * 1024 * 1024 * 1024:  # &gt;100GB
++             # Very large sequences: use 1% of size, capped at 4GB
++             input_buffer = min(int(total_bytes * 0.01), 4 * 1024 * 1024 * 1024)
++             overhead = 1 * 1024 * 1024 * 1024  # 1GB overhead
+          else:
+-             # Smaller sequences: use 15% of size, capped at 2GB
+-             input_buffer = min(int(total_bytes * 0.15), 2 * 1024 * 1024 * 1024)
+-             overhead = 1 * 1024 * 1024 * 1024  # 1GB for smaller sequences
++             # Normal sequences: use small fixed buffer
++             input_buffer = 512 * 1024 * 1024  # 512MB buffer
++             overhead = 512 * 1024 * 1024  # 512MB overhead
+          
+          # Total RAM estimate
+```
+
+</details>
+
+</tool-use>
+
+---
+
+
+
+---
+
+<tool-use data-tool-type="generic" data-tool-name="read_lints">
+<details>
+          <summary>Tool use: **read_lints** • Read lints for 1 file</summary>
+      
+Lint paths:
+
+- `zip_sequences.py`
+
+Lint results
+
+
+**No lint errors found**
+</details>
+</tool-use>
+
+---
+
+
+
+
+
+---
+
+
+
+---
+
+
+Updated the RAM estimation to match observed usage. Changes:
+
+1. **Reduced dictionary RAM multiplier**: From 11x to 8x (8GB instead of 11GB for a 1GB dictionary), matching observed 7–10GB usage
+2. **Reduced input buffer**: From 15% of sequence size (capped at 2GB) to a fixed 512MB for normal sequences
+3. **Reduced overhead**: From 1GB to 512MB for normal sequences
+
+**New estimate for a 1GB sequence**: 8GB + 512MB + 512MB = **9GB** (down from ~12GB)
+
+This aligns with observed usage and allows more concurrent workers since the bin-packing algorithm can fit more jobs into the available RAM. For your 15 sequences totaling ~12GB, the estimate should be around 135GB total (15 × 9GB) instead of 181.6GB, allowing more workers to run concurrently.
+
+---
+
+_**User (2025-11-12 16:48Z)**_
+
+It's still only giving 2 instances that are allocating around 2gb. We should be more aggressive or tighten our estimation.
+
+PS P:\250827_FestivalTurf\Renders> & P:\250827_FestivalTurf\Renders\ZipSeqArchv.bat
+
+[init] Loading configuration sources...
+
+[init] Reading project config at P:\250827_FestivalTurf\.config\config.json
+
+[init] Configuration loaded from project
+
+[init] zip_sequences starting up...
+
+[init] Working directory: P:\250827_FestivalTurf
+
+[init] Mode: zip, zipper: 7z, jobs arg: auto
+
+[init] 7z executable: C:\ProgramData\chocolatey\bin\7z.EXE, Max7zInst: auto
+
+[init] Scanning sequences under \\NEXUS\proj\250827_FestivalTurf\Renders
+
+[scan] [1] Inspecting 01 Develop Your Turf Plan\01_Blueprint
+
+[scan] 01 Develop Your Turf Plan\01_Blueprint metadata unchanged; archive up to date
+
+[scan] [2] Inspecting 01 Develop Your Turf Plan\01_Blueprint_insert
+
+[scan] 01 Develop Your Turf Plan\01_Blueprint_insert metadata unchanged; archive up to date
+
+[scan] [3] Inspecting 01 Develop Your Turf Plan\02_Roll Sizes
+
+[scan] 01 Develop Your Turf Plan\02_Roll Sizes metadata unchanged; archive up to date
+
+[scan] [4] Inspecting 01 Develop Your Turf Plan\03_Window
+
+[scan] 01 Develop Your Turf Plan\03_Window metadata unchanged; archive up to date
+
+[scan] [5] Inspecting 01 Develop Your Turf Plan\04_Dog
+
+[scan] 01 Develop Your Turf Plan\04_Dog metadata unchanged; archive up to date
+
+[scan] [10] Inspecting 02 Preparing the Base\01_intro
+
+[scan] [20] Inspecting 02 Preparing the Base\10_FT_Talking
+
+[scan] [30] Inspecting 03 Roll Out And Stretch\05_diagram
+
+[scan] Skipping 04 Securing Your Seam\02_FT talking (unchanged since archive)
+
+[scan] Skipping 04 Securing Your Seam\02_FT talking_Pt2 (unchanged since archive)
+
+[scan] [40] Inspecting 04 Securing Your Seam\02_S seam insert
+
+[scan] Skipping 04 Securing Your Seam\04_holding nails (unchanged since archive)
+
+[scan] [50] Inspecting 04 Securing Your Seam\Nail_dimpling_example
+
+[scan] Queued 05 Stretch Cut Nail\01_intro for compression (~1.10GB) [1 total]
+
+[scan] Queued 05 Stretch Cut Nail\02_kicker insert for compression (~0.24GB) [2 total]
+
+[scan] Queued 05 Stretch Cut Nail\02_talking for compression (~0.81GB) [3 total]
+
+[scan] Queued 05 Stretch Cut Nail\03_FT Shuffle for compression (~0.70GB) [4 total]
+
+[scan] Queued 05 Stretch Cut Nail\03_FT Shuffle_dance_only for compression (~0.38GB) [5 total]
+
+[scan] [60] Inspecting 05 Stretch Cut Nail\Visual 10
+
+[scan] Queued 05 Stretch Cut Nail\Visual 10 for compression (~1.28GB) [10 total]
+
+[scan] Queued 05 Stretch Cut Nail\Visual 8 for compression (~0.94GB) [15 total]
+
+[zip] RAM: 102.2GB available (80% of total), 97.1GB for compression (95%)
+
+[zip] Estimated RAM per sequence: 144.0GB total across 16 sequences
+
+[zip] Bin-packing: 2 workers, bin sizes: ['90.0GB', '54.0GB']
+
+[zip] Final worker count: 2 (requested: None)
+
+[init] Preparing to compress 16 sequence(s) with 2 worker(s)
+
+[zip] Renders\05 Stretch Cut Nail\01_intro -> Renders\_zipped\05 Stretch Cut Nail\01_intro.7z
+
+[zip] Renders\05 Stretch Cut Nail\02_kicker insert -> Renders\_zipped\05 Stretch Cut Nail\02_kicker insert.7z
+
+[zip] Renders\05 Stretch Cut Nail\02_talking -> Renders\_zipped\05 Stretch Cut Nail\02_talking.7z
+
+[zip] 1/16 05 Stretch Cut Nail\02_kicker insert
+
+---
+
+_**Agent (model default, mode Agent)**_
+
+
+
+---
+
+
+Updating the RAM estimate to match actual usage (~2GB per process) so we can run more workers:
+
+
+---
+
+<tool-use data-tool-type="read" data-tool-name="read_file">
+<details><summary>Tool use: **read_file** • Read file: zip_sequences.py</summary>
+
+</details>
+</tool-use>
+
+---
+
+
+
+---
+
+
+Tightening the RAM estimate to match actual usage (~2GB per process):
+
+
+---
+
+<tool-use data-tool-type="write" data-tool-name="search_replace">
+
+<details><summary>Tool use: **code_edit** • Edit file: zip_sequences.py</summary>
+
+**Chunk 1**
+Lines added: 10, lines removed: 19
+
+```diff
+@@ -211,24 +211,15 @@
+      if ZIPPER_TYPE == &quot;7z&quot;:
+          # Fixed dictionary size: 1GB (1024MB)
+-         FIXED_DICT_SIZE_MB = 1024
+-         FIXED_DICT_SIZE_BYTES = FIXED_DICT_SIZE_MB * 1024 * 1024
+-         
+-         # Dictionary RAM: ~8x dictionary size (1GB dict = 8GB RAM)
+-         # Based on actual observed usage: 7z processes use ~7-10GB with 1GB dictionary
+-         dict_ram = FIXED_DICT_SIZE_BYTES * 8  # 8GB for 1GB dictionary
+-         
+-         # Input buffer: 7z processes files in chunks, minimal buffer needed
+-         # For most sequences, a small fixed buffer is sufficient
+-         if total_bytes &gt; 100 * 1024 * 1024 * 1024:  # &gt;100GB
+-             # Very large sequences: use 1% of size, capped at 4GB
+-             input_buffer = min(int(total_bytes * 0.01), 4 * 1024 * 1024 * 1024)
+-             overhead = 1 * 1024 * 1024 * 1024  # 1GB overhead
++         # Actual observed usage: ~2GB per process regardless of sequence size
++         # Use a tight estimate to allow maximum concurrency
++         # Estimate 3GB per job (50% safety margin over observed ~2GB)
++         base_ram = 3 * 1024 * 1024 * 1024  # 3GB base estimate
++         
++         # For very large sequences (&gt;50GB), add small buffer
++         if total_bytes &gt; 50 * 1024 * 1024 * 1024:  # &gt;50GB
++             # Add 1GB for very large sequences
++             estimated_ram = base_ram + (1 * 1024 * 1024 * 1024)
+          else:
+-             # Normal sequences: use small fixed buffer
+-             input_buffer = 512 * 1024 * 1024  # 512MB buffer
+-             overhead = 512 * 1024 * 1024  # 512MB overhead
+-         
+-         # Total RAM estimate
+-         estimated_ram = dict_ram + input_buffer + overhead
++             estimated_ram = base_ram
+          
+          return estimated_ram
+```
+
+</details>
+
+</tool-use>
+
+---
+
+
+
+---
+
+<tool-use data-tool-type="generic" data-tool-name="read_lints">
+<details>
+          <summary>Tool use: **read_lints** • Read lints for 1 file</summary>
+      
+Lint paths:
+
+- `zip_sequences.py`
+
+Lint results
+
+
+**No lint errors found**
+</details>
+</tool-use>
+
+---
+
+
+
+
+
+---
+
+
+
+---
+
+
+Updated the RAM estimate to match actual usage:
+
+**Changes:**
+1. **Reduced estimate from 9GB to 3GB per job** — based on observed ~2GB usage with a 50% safety margin
+2. **Simplified calculation** — removed the dictionary multiplier; using a fixed 3GB base estimate
+3. **Only add buffer for very large sequences** — sequences >50GB get +1GB
+
+**Impact:**
+- **Before**: 16 sequences × 9GB = 144GB estimated → 2 workers
+- **After**: 16 sequences × 3GB = 48GB estimated → ~32 workers possible
+
+With 97GB available for compression and 3GB per job, bin-packing should allow many more concurrent workers. The estimate is tighter while still leaving headroom for actual usage variations.
+
+---
+
+_**User (2025-11-12 16:48Z)**_
+
+with this estimate, it should be able to work on all of them concurrently. why is it only doing one?
+
+PS P:\250827_FestivalTurf\Renders> & P:\250827_FestivalTurf\Renders\ZipSeqArchv.bat
+
+[init] Loading configuration sources...
+
+[init] Reading project config at P:\250827_FestivalTurf\.config\config.json
+
+[init] Configuration loaded from project
+
+[init] zip_sequences starting up...
+
+[init] Working directory: P:\250827_FestivalTurf
+
+[init] Mode: zip, zipper: 7z, jobs arg: auto
+
+[init] 7z executable: C:\ProgramData\chocolatey\bin\7z.EXE, Max7zInst: auto
+
+[init] Scanning sequences under \\NEXUS\proj\250827_FestivalTurf\Renders
+
+[scan] [1] Inspecting 01 Develop Your Turf Plan\01_Blueprint
+
+[scan] 01 Develop Your Turf Plan\01_Blueprint metadata unchanged; archive up to date
+
+[scan] [2] Inspecting 01 Develop Your Turf Plan\01_Blueprint_insert
+
+[scan] 01 Develop Your Turf Plan\01_Blueprint_insert metadata unchanged; archive up to date
+
+[scan] [3] Inspecting 01 Develop Your Turf Plan\02_Roll Sizes
+
+[scan] 01 Develop Your Turf Plan\02_Roll Sizes metadata unchanged; archive up to date
+
+[scan] [4] Inspecting 01 Develop Your Turf Plan\03_Window
+
+[scan] 01 Develop Your Turf Plan\03_Window metadata unchanged; archive up to date
+
+[scan] [5] Inspecting 01 Develop Your Turf Plan\04_Dog
+
+[scan] 01 Develop Your Turf Plan\04_Dog metadata unchanged; archive up to date
+
+[scan] [10] Inspecting 02 Preparing the Base\01_intro
+
+[scan] [20] Inspecting 02 Preparing the Base\10_FT_Talking
+
+[scan] [30] Inspecting 03 Roll Out And Stretch\05_diagram
+
+[scan] Skipping 04 Securing Your Seam\02_FT talking (unchanged since archive)
+
+[scan] Skipping 04 Securing Your Seam\02_FT talking_Pt2 (unchanged since archive)
+
+[scan] [40] Inspecting 04 Securing Your Seam\02_S seam insert
+
+[scan] Skipping 04 Securing Your Seam\04_holding nails (unchanged since archive)
+
+[scan] [50] Inspecting 04 Securing Your Seam\Nail_dimpling_example
+
+[scan] Queued 05 Stretch Cut Nail\01_intro for compression (~1.10GB) [1 total]
+
+[scan] Skipping 05 Stretch Cut Nail\02_kicker insert (unchanged since archive)
+
+[scan] Queued 05 Stretch Cut Nail\02_talking for compression (~0.81GB) [2 total]
+
+[scan] Queued 05 Stretch Cut Nail\03_FT Shuffle for compression (~0.70GB) [3 total]
+
+[scan] Queued 05 Stretch Cut Nail\03_FT Shuffle_dance_only for compression (~0.38GB) [4 total]
+
+[scan] Queued 05 Stretch Cut Nail\03_FT Shuffle_Intro for compression (~0.46GB) [5 total]
+
+[scan] [60] Inspecting 05 Stretch Cut Nail\Visual 10
+
+[scan] Queued 05 Stretch Cut Nail\Visual 11 for compression (~1.17GB) [10 total]
+
+[scan] Queued 05 Stretch Cut Nail\Visual 9 for compression (~0.37GB) [15 total]
+
+[zip] RAM: 102.2GB available (80% of total), 97.1GB for compression (95%)
+
+[zip] Estimated RAM per sequence: 45.0GB total across 15 sequences
+
+[zip] Bin-packing: 1 workers, bin sizes: ['45.0GB']
+
+[zip] Final worker count: 1 (requested: None)
+
+[init] Preparing to compress 15 sequence(s) with 1 worker(s)
+
+[zip] Renders\05 Stretch Cut Nail\01_intro -> Renders\_zipped\05 Stretch Cut Nail\01_intro.7z
+
+---
+
+_**Agent (model default, mode Agent)**_
+
+
+
+---
+
+
+The bin-packing algorithm is packing all sequences into a single bin. With 97GB available and 3GB per sequence, we should run all 15 concurrently. Checking the bin-packing logic:
+
+
+---
+
+<tool-use data-tool-type="read" data-tool-name="read_file">
+<details><summary>Tool use: **read_file** • Read file: zip_sequences.py</summary>
+
+</details>
+</tool-use>
+
+---
+
+
+
+---
+
+
+The bin-packing minimizes bins (workers), so it packs all sequences into one bin. For maximum concurrency, we should run one worker per sequence when they all fit. Updating the logic:
+
+
+---
+
+<tool-use data-tool-type="write" data-tool-name="search_replace">
+
+<details><summary>Tool use: **code_edit** • Edit file: zip_sequences.py</summary>
+
+**Chunk 1**
+Lines added: 2, lines removed: 2
+
+```diff
+@@ -299,11 +299,11 @@
+          
+          # Auto-calculate based on RAM if Max7zInst not configured
+-         # Use bin-packing algorithm with size-aware RAM estimation
++         # Prioritize maximum concurrency: one worker per sequence when possible
+          if available_ram is not None:
+              # available_ram is already 80% of total (20% reserved for system)
+              # Use 95% of available RAM for compression jobs
+              compression_ram = int(available_ram * 0.95)
+              
+-             # Estimate RAM for each work item and create list with (seq_dir, estimated_ram, work_item)
++             # Estimate RAM for each work item
+              work_items_with_ram: list[tuple[Path, int, tuple[Path, Path, Path, dict]]] = []
+              ram_limits_dict: dict[Path, int] = {}
+```
+
+**Chunk 2**
+Lines added: 36, lines removed: 27
+
+```diff
+@@ -315,36 +315,45 @@
+                      ram_limits_dict[seq_dir] = estimated_ram
+                  except Exception:
+-                     # If estimation fails, use a safe default (12GB minimum for 1GB dict)
+-                     default_ram = 12 * 1024 * 1024 * 1024  # 12GB
++                     # If estimation fails, use a safe default (3GB)
++                     default_ram = 3 * 1024 * 1024 * 1024  # 3GB
+                      work_items_with_ram.append((seq_dir, default_ram, (seq_dir, zip_path, state_path, seq_state)))
+                      ram_limits_dict[seq_dir] = default_ram
+              
+-             # Sort by estimated RAM (largest first) for bin-packing
+-             work_items_with_ram.sort(key=lambda x: x[1], reverse=True)
+-             
+-             # Bin-packing algorithm: pack largest items first
+-             bins: list[list[tuple[Path, int, tuple[Path, Path, Path, dict]]]] = []
+-             bin_remaining: list[int] = []
+-             
+-             for seq_dir, estimated_ram, work_item in work_items_with_ram:
+-                 # Try to fit in existing bin
+-                 placed = False
+-                 for i, remaining in enumerate(bin_remaining):
+-                     if remaining &gt;= estimated_ram:
+-                         bins[i].append((seq_dir, estimated_ram, work_item))
+-                         bin_remaining[i] -= estimated_ram
+-                         placed = True
+-                         break
+-                 
+-                 # If doesn&#39;t fit, create new bin
+-                 if not placed:
+-                     bins.append([(seq_dir, estimated_ram, work_item)])
+-                     bin_remaining.append(compression_ram - estimated_ram)
+-             
+-             # Worker count is number of bins
+-             worker_count = len(bins)
++             num_work_items = len(work_items) if work_items else 0
++             
++             # Calculate total estimated RAM
++             total_estimated_ram = sum(ram for _, ram, _ in work_items_with_ram)
++             
++             # If all sequences fit in available RAM, use one worker per sequence (maximum concurrency)
++             if total_estimated_ram &lt;= compression_ram:
++                 worker_count = num_work_items
++             else:
++                 # Not all fit - use bin-packing to minimize workers
++                 # Sort by estimated RAM (largest first) for bin-packing
++                 work_items_with_ram.sort(key=lambda x: x[1], reverse=True)
++                 
++                 # Bin-packing algorithm: pack largest items first
++                 bins: list[list[tuple[Path, int, tuple[Path, Path, Path, dict]]]] = []
++                 bin_remaining: list[int] = []
++                 
++                 for seq_dir, estimated_ram, work_item in work_items_with_ram:
++                     # Try to fit in existing bin
++                     placed = False
++                     for i, remaining in enumerate(bin_remaining):
++                         if remaining &gt;= estimated_ram:
++                             bins[i].append((seq_dir, estimated_ram, work_item))
++                             bin_remaining[i] -= estimated_ram
++                             placed = True
++                             break
++                     
++                     # If doesn&#39;t fit, create new bin
++                     if not placed:
++                         bins.append([(seq_dir, estimated_ram, work_item)])
++                         bin_remaining.append(compression_ram - estimated_ram)
++                 
++                 # Worker count is number of bins
++                 worker_count = len(bins)
+              
+              # Cap at number of actual work items (can&#39;t have more workers than jobs)
+-             num_work_items = len(work_items) if work_items else 0
+              if num_work_items &gt; 0:
+                  worker_count = min(worker_count, num_work_items)
+```
+
+**Chunk 3**
+Lines added: 1, lines removed: 1
+
+```diff
+@@ -357,5 +366,5 @@
+                  ram_gb = available_ram / (1024 ** 3)
+                  compression_ram_gb = compression_ram / (1024 ** 3)
+-                 total_estimated_gb = sum(ram for _, ram, _ in work_items_with_ram) / (1024 ** 3)
++                 total_estimated_gb = total_estimated_ram / (1024 ** 3)
+                  log(
+                      &quot;zip&quot;,
+```
+
+**Chunk 4**
+Lines added: 9, lines removed: 3
+
+```diff
+@@ -370,12 +379,18 @@
+                      verbose=verbose
+                  )
+-                 if len(bins) &gt; 0:
+-                     bin_sizes = [sum(ram for _, ram, _ in bin_items) / (1024 ** 3) for bin_items in bins]
++                 if total_estimated_ram &lt;= compression_ram:
+                      log(
+                          &quot;zip&quot;,
+-                         f&quot;Bin-packing: {worker_count} workers, bin sizes: {[f&#39;{s:.1f}GB&#39; for s in bin_sizes[:5]]}{&#39;...&#39; if len(bin_sizes) &gt; 5 else &#39;&#39;}&quot;,
++                         f&quot;All sequences fit in RAM → using {worker_count} workers (one per sequence)&quot;,
+                          verbose_only=True,
+                          verbose=verbose
+                      )
++                 else:
++                     log(
++                         &quot;zip&quot;,
++                         f&quot;Using bin-packing: {worker_count} workers needed&quot;,
++                         verbose_only=True,
++                         verbose=verbose
++                     )
+                  log(
+                      &quot;zip&quot;,
+```
+
+</details>
+
+</tool-use>
+
+---
+
+
+
+---
+
+<tool-use data-tool-type="generic" data-tool-name="read_lints">
+<details>
+          <summary>Tool use: **read_lints** • Read lints for 1 file</summary>
+      
+Lint paths:
+
+- `zip_sequences.py`
+
+Lint results
+
+
+**No lint errors found**
+</details>
+</tool-use>
+
+---
+
+
+
+
+
+---
+
+
+
+---
+
+
+Updated the logic to prioritize maximum concurrency. Changes:
+
+1. **Check if all sequences fit**: If total estimated RAM (45GB) ≤ available compression RAM (97GB), use one worker per sequence.
+2. **Use bin-packing only when needed**: Only when sequences don't all fit in RAM.
+
+**Result:**
+- **Before**: Bin-packing minimized bins → 1 worker for all 15 sequences
+- **After**: All sequences fit → 15 workers (one per sequence), all running concurrently
+
+With 45GB total estimated RAM and 97GB available, all 15 sequences will run concurrently instead of sequentially.
+
+---
+
+_**User (2025-11-12 16:48Z)**_
+
+what is bin packing?
+
+---
+
+_**Agent (model default, mode Agent)**_
+
+
+
+---
+
+
+**Bin packing** is an optimization problem: given items of different sizes and bins of fixed capacity, pack all items using as few bins as possible.
+
+## In Your Code Context
+
+In the compression script, bin packing is used to:
+- **Items**: Sequences to compress (each estimated at ~3GB RAM)
+- **Bins**: Workers (each can use up to 97GB RAM)
+- **Goal**: Minimize the number of workers needed
+
+### Example:
+If you have 15 sequences (each ~3GB) and 97GB available per worker:
+- **Without bin packing**: Could create 15 workers (one per sequence)
+- **With bin packing**: Packs sequences into as few workers as possible
+  - Worker 1: Sequences 1-32 (96GB total)
+  - Worker 2: Sequences 33-64 (96GB total)
+  - etc.
+
+## Why It Was a Problem
+
+Bin packing minimizes the number of workers, which can reduce concurrency. With 15 sequences totaling 45GB and 97GB available, bin packing put all 15 into one worker, so they ran sequentially.
+
+## The Fix
+
+The code now checks if all sequences fit in available RAM:
+- If yes: Use one worker per sequence (maximum concurrency)
+- If no: Use bin packing to minimize workers
+
+This prioritizes speed when possible, and efficiency when needed.
+
+---
+

zip_sequences.py

@@ -58,6 +58,12 @@ DEFAULT_CONFIG = {
 }
 
 
+def log(mode: str, message: str, *, verbose_only: bool = False, verbose: bool = False) -> None:
+    if verbose_only and not verbose:
+        return
+    print(f"[{mode}] {message}", flush=True)
+
+
 def load_config() -> dict:
     # First try to load from project's .config folder (current working directory)
     # Then fall back to ProjectStructure repo config (next to zip_sequences.py)
@@ -70,22 +76,28 @@ def load_config() -> dict:
         ("repo", repo_config),
     ]
     
+    log("init", "Loading configuration sources...")
     for source, config_path in config_paths:
         try:
             if config_path.exists():
+                log("init", f"Reading {source} config at {config_path}")
                 text = config_path.read_text(encoding="utf-8")
                 try:
                     data = json.loads(text)
                     if isinstance(data, dict):
                         merged = DEFAULT_CONFIG.copy()
                         merged.update(data)
+                        log("init", f"Configuration loaded from {source}")
                         return merged
                 except json.JSONDecodeError:
+                    log("init", f"Config file at {config_path} is invalid JSON; skipping")
                     continue
         except OSError:
+            log("init", f"Unable to read config at {config_path}; skipping")
             continue
     
     # If no config found, return defaults
+    log("init", "No config files found; using default settings")
     return DEFAULT_CONFIG.copy()
 
 
@@ -197,22 +209,18 @@ def estimate_ram_per_job(seq_dir: Path, seq_state: dict) -> int:
     total_bytes = sum(entry.get("size", 0) for entry in seq_state.get("files", []))
     
     if ZIPPER_TYPE == "7z":
-        # Base RAM: 500MB per job
+        # Fixed dictionary size: 1GB (1024MB)
-        base_ram = 500 * 1024 * 1024  # 500 MB
+        # Actual observed usage: ~2GB per process regardless of sequence size
+        # Use a tight estimate to allow maximum concurrency
+        # Estimate 3GB per job (50% safety margin over observed ~2GB)
+        base_ram = 3 * 1024 * 1024 * 1024  # 3GB base estimate
         
-        # Compression factor: 7z can use significant RAM, especially for large files
+        # For very large sequences (>50GB), add small buffer
-        # Use 0.15x factor (conservative estimate accounting for 7z's 80% usage)
+        if total_bytes > 50 * 1024 * 1024 * 1024:  # >50GB
-        compression_factor = 0.15
+            # Add 1GB for very large sequences
-        
+            estimated_ram = base_ram + (1 * 1024 * 1024 * 1024)
-        # For very large folders (>10GB), cap at 8GB per job
-        max_ram_per_job = 8 * 1024 * 1024 * 1024  # 8 GB
-        large_folder_threshold = 10 * 1024 * 1024 * 1024  # 10 GB
-        
-        if total_bytes > large_folder_threshold:
-            estimated_ram = max_ram_per_job
         else:
-            estimated_ram = max(base_ram, int(total_bytes * compression_factor))
+            estimated_ram = base_ram
-            estimated_ram = min(estimated_ram, max_ram_per_job)
         
         return estimated_ram
     else:
@@ -245,7 +253,7 @@ def max_workers(
     
     # If no work items provided, return CPU-based limit
     if work_items is None or len(work_items) == 0:
-        return (cpu_limit, None)
+        return (cpu_limit, {})
     
     # Try to calculate RAM-based limit
     available_ram = get_available_ram()
@@ -254,7 +262,7 @@ def max_workers(
         # RAM detection failed, fall back to CPU limit
         if verbose:
             log("zip", "RAM detection failed, using CPU-based worker limit", verbose_only=True, verbose=verbose)
-        return (cpu_limit, None)
+        return (cpu_limit, {})
     
     # For 7z: use fixed dictionary size and calculate workers
     if ZIPPER_TYPE == "7z":
@@ -272,6 +280,12 @@ def max_workers(
             if requested and requested > 0:
                 final_limit = min(final_limit, requested)
             
+            # Create RAM limits dict (all use fixed dict size, but return as dict for consistency)
+            ram_limits_dict: dict[Path, int] = {}
+            if work_items:
+                for seq_dir, _, _, _ in work_items:
+                    ram_limits_dict[seq_dir] = fixed_dict_size_bytes
+            
             if verbose:
                 log(
                     "zip",
@@ -281,45 +295,112 @@ def max_workers(
                     verbose=verbose
                 )
             
-            return (final_limit, fixed_dict_size_bytes)
+            return (final_limit, ram_limits_dict)
         
         # Auto-calculate based on RAM if Max7zInst not configured
-        # "Balls-to-the-walls" mode: use maximum resources
+        # Prioritize maximum concurrency: one worker per sequence when possible
         if available_ram is not None:
-            # 7z uses ~2-3x dictionary size in RAM, use 3x for aggressive mode
-            # This is more realistic and allows more concurrent workers
-            FIXED_RAM_PER_JOB = FIXED_DICT_SIZE_MB * 3 * 1024 * 1024  # 3GB per job
-            
             # available_ram is already 80% of total (20% reserved for system)
-            # Use 95% of available RAM for compression jobs (aggressive mode)
+            # Use 95% of available RAM for compression jobs
             compression_ram = int(available_ram * 0.95)
             
-            # Calculate worker limit based on fixed per-job RAM
+            # Estimate RAM for each work item
-            ram_limit = max(1, compression_ram // FIXED_RAM_PER_JOB)
+            work_items_with_ram: list[tuple[Path, int, tuple[Path, Path, Path, dict]]] = []
+            ram_limits_dict: dict[Path, int] = {}
+            
+            for seq_dir, zip_path, state_path, seq_state in work_items:
+                try:
+                    estimated_ram = estimate_ram_per_job(seq_dir, seq_state)
+                    work_items_with_ram.append((seq_dir, estimated_ram, (seq_dir, zip_path, state_path, seq_state)))
+                    ram_limits_dict[seq_dir] = estimated_ram
+                except Exception:
+                    # If estimation fails, use a safe default (3GB)
+                    default_ram = 3 * 1024 * 1024 * 1024  # 3GB
+                    work_items_with_ram.append((seq_dir, default_ram, (seq_dir, zip_path, state_path, seq_state)))
+                    ram_limits_dict[seq_dir] = default_ram
+            
+            num_work_items = len(work_items) if work_items else 0
+            
+            # Calculate total estimated RAM
+            total_estimated_ram = sum(ram for _, ram, _ in work_items_with_ram)
+            
+            # If all sequences fit in available RAM, use one worker per sequence (maximum concurrency)
+            if total_estimated_ram <= compression_ram:
+                worker_count = num_work_items
+            else:
+                # Not all fit - use bin-packing to minimize workers
+                # Sort by estimated RAM (largest first) for bin-packing
+                work_items_with_ram.sort(key=lambda x: x[1], reverse=True)
+                
+                # Bin-packing algorithm: pack largest items first
+                bins: list[list[tuple[Path, int, tuple[Path, Path, Path, dict]]]] = []
+                bin_remaining: list[int] = []
+                
+                for seq_dir, estimated_ram, work_item in work_items_with_ram:
+                    # Try to fit in existing bin
+                    placed = False
+                    for i, remaining in enumerate(bin_remaining):
+                        if remaining >= estimated_ram:
+                            bins[i].append((seq_dir, estimated_ram, work_item))
+                            bin_remaining[i] -= estimated_ram
+                            placed = True
+                            break
+                    
+                    # If doesn't fit, create new bin
+                    if not placed:
+                        bins.append([(seq_dir, estimated_ram, work_item)])
+                        bin_remaining.append(compression_ram - estimated_ram)
+                
+                # Worker count is number of bins
+                worker_count = len(bins)
             
             # Cap at number of actual work items (can't have more workers than jobs)
-            num_work_items = len(work_items) if work_items else 0
             if num_work_items > 0:
-                ram_limit = min(ram_limit, num_work_items)
+                worker_count = min(worker_count, num_work_items)
             
-            # Use RAM limit directly (no CPU limit)
+            # Respect user's --jobs if provided
-            final_limit = ram_limit
             if requested and requested > 0:
-                final_limit = min(final_limit, requested)
+                worker_count = min(worker_count, requested)
             
             if verbose:
                 ram_gb = available_ram / (1024 ** 3)
                 compression_ram_gb = compression_ram / (1024 ** 3)
-                ram_per_job_gb = FIXED_RAM_PER_JOB / (1024 ** 3)
+                total_estimated_gb = total_estimated_ram / (1024 ** 3)
                 log(
                     "zip",
-                    f"RAM: {ram_gb:.1f}GB available (80% of total), {compression_ram_gb:.1f}GB for compression (95%), {ram_per_job_gb:.1f}GB per job (dict: {FIXED_DICT_SIZE_MB}MB) → "
+                    f"RAM: {ram_gb:.1f}GB available (80% of total), {compression_ram_gb:.1f}GB for compression (95%)",
-                    f"RAM limit: {ram_limit}, work items: {num_work_items}, requested: {requested}, final: {final_limit}",
+                    verbose_only=True,
+                    verbose=verbose
+                )
+                log(
+                    "zip",
+                    f"Estimated RAM per sequence: {total_estimated_gb:.1f}GB total across {num_work_items} sequences",
+                    verbose_only=True,
+                    verbose=verbose
+                )
+                if total_estimated_ram <= compression_ram:
+                    log(
+                        "zip",
+                        f"All sequences fit in RAM → using {worker_count} workers (one per sequence)",
+                        verbose_only=True,
+                        verbose=verbose
+                    )
+                else:
+                    log(
+                        "zip",
+                        f"Using bin-packing: {worker_count} workers needed",
+                        verbose_only=True,
+                        verbose=verbose
+                    )
+                log(
+                    "zip",
+                    f"Final worker count: {worker_count} (requested: {requested})",
                     verbose_only=True,
                     verbose=verbose
                 )
             
-            return (final_limit, fixed_dict_size_bytes)
+            # Return worker count and RAM limits dict
+            return (worker_count, ram_limits_dict)
         
         # RAM detection failed, use a safe default (no CPU limit)
         default_limit = 4
@@ -328,6 +409,14 @@ def max_workers(
             default_limit = min(default_limit, num_work_items)
         if requested and requested > 0:
             default_limit = min(default_limit, requested)
+        
+        # Create RAM limits dict with safe defaults (12GB per job for 1GB dict)
+        ram_limits_dict: dict[Path, int] = {}
+        if work_items:
+            default_ram = 12 * 1024 * 1024 * 1024  # 12GB default
+            for seq_dir, _, _, _ in work_items:
+                ram_limits_dict[seq_dir] = default_ram
+        
         if verbose:
             log(
                 "zip",
@@ -336,21 +425,25 @@ def max_workers(
                 verbose_only=True,
                 verbose=verbose
             )
-        return (default_limit, fixed_dict_size_bytes)
+        return (default_limit, ram_limits_dict)
     
     # For zip compression, use existing estimation-based approach
     # Estimate RAM per job for each work item
     ram_estimates = []
+    ram_limits_dict: dict[Path, int] = {}
     for seq_dir, zip_path, state_path, seq_state in work_items:
         try:
             estimated_ram = estimate_ram_per_job(seq_dir, seq_state)
             ram_estimates.append(estimated_ram)
+            ram_limits_dict[seq_dir] = estimated_ram
         except Exception:
             # If estimation fails, use fallback estimate
-            ram_estimates.append(1024 * 1024 * 1024)  # 1GB fallback for zip
+            fallback_ram = 1024 * 1024 * 1024  # 1GB fallback for zip
+            ram_estimates.append(fallback_ram)
+            ram_limits_dict[seq_dir] = fallback_ram
     
     if not ram_estimates:
-        return (cpu_limit, None)
+        return (cpu_limit, {})
     
     max_ram_per_job = max(ram_estimates)
     ram_limit = max(1, available_ram // max_ram_per_job)
@@ -368,13 +461,7 @@ def max_workers(
             verbose=verbose
         )
     
-    return (final_limit, None)
+    return (final_limit, ram_limits_dict)
-
-
-def log(mode: str, message: str, *, verbose_only: bool = False, verbose: bool = False) -> None:
-    if verbose_only and not verbose:
-        return
-    print(f"[{mode}] {message}")
 
 
 def is_archive_path(path: Path) -> bool:
@@ -516,11 +603,11 @@ def zip_sequence(seq_dir: Path, zip_path: Path, per_job_memory_limit: int | None
                 "-t7z",  # Use 7z format, not zip
             ]
             
-            # Add fixed dictionary size if specified (7z memory usage is controlled by dictionary size)
+            # Always use fixed dictionary size: 1GB (1024MB)
-            if per_job_memory_limit is not None:
+            # The per_job_memory_limit parameter is the estimated RAM usage (for logging/info only)
-                # per_job_memory_limit is actually the fixed dictionary size in bytes
+            # We keep dictionary at 1GB for best compression regardless of RAM estimate
-                dict_size_mb = per_job_memory_limit // (1024 * 1024)
+            FIXED_DICT_SIZE_MB = 1024
-                cmd.append(f"-md={dict_size_mb}m")
+            cmd.append(f"-md={FIXED_DICT_SIZE_MB}m")
             
             cmd.extend([
                 str(temp_zip_abs),
@@ -655,9 +742,20 @@ def process_expand(zip_path: Path, state: dict, *, verbose: bool) -> None:
 
 def run_zip(requested_workers: int | None, *, verbose: bool) -> int:
     work_items: list[tuple[Path, Path, Path, dict]] = []
+    log("init", f"Scanning sequences under {RENDER_ROOT.resolve()}")
+    total_scanned = 0
+    quick_skipped = 0
+    state_skipped = 0
+    empty_dirs = 0
+    queued = 0
 
     if RENDER_ROOT.exists():
         for seq_dir in find_sequence_dirs(RENDER_ROOT):
+            total_scanned += 1
+            rel = seq_dir.relative_to(RENDER_ROOT)
+            if total_scanned <= 5 or total_scanned % 10 == 0:
+                log("scan", f"[{total_scanned}] Inspecting {rel}")
+
             # Get the target archive path (will be .7z if ZIPPER_TYPE is "7z")
             zip_path = archive_path_for(seq_dir)
             state_path = state_path_for(zip_path)
@@ -685,6 +783,9 @@ def run_zip(requested_workers: int | None, *, verbose: bool) -> int:
                     archive_mtime = zip_path.stat().st_mtime_ns
                     # If directory wasn't modified since archive was created, skip state computation
                     if dir_mtime <= archive_mtime:
+                        quick_skipped += 1
+                        if quick_skipped <= 5:
+                            log("scan", f"Skipping {rel} (unchanged since archive)")
                         # Still need to check for old .zip cleanup (we have .7z, so .zip is obsolete)
                         if old_zip_path and old_zip_path.exists():
                             old_zip_path.unlink(missing_ok=True)
@@ -699,11 +800,17 @@ def run_zip(requested_workers: int | None, *, verbose: bool) -> int:
             # Compute current state only if we need to
             seq_state = compute_state(seq_dir)
             if not seq_state["files"]:
+                empty_dirs += 1
+                if empty_dirs <= 5:
+                    log("scan", f"{rel} has no files; skipping")
                 continue
             
             # Check if state changed
             if stored_state is not None and not state_changed(seq_state, stored_state):
                 # Metadata matches stored state
+                state_skipped += 1
+                if state_skipped <= 5:
+                    log("scan", f"{rel} metadata unchanged; archive up to date")
                 if zip_path.exists():
                     # .7z exists and is up to date, clean up old .zip if it exists
                     if old_zip_path and old_zip_path.exists():
@@ -720,16 +827,30 @@ def run_zip(requested_workers: int | None, *, verbose: bool) -> int:
                     continue
 
             work_items.append((seq_dir, zip_path, state_path, seq_state))
+            queued += 1
+            if queued <= 5 or queued % 5 == 0:
+                total_bytes = sum(entry.get("size", 0) for entry in seq_state.get("files", []))
+                size_gb = total_bytes / (1024 ** 3)
+                log("scan", f"Queued {rel} for compression (~{size_gb:.2f}GB) [{queued} total]")
 
     if not work_items:
         if not RENDER_ROOT.exists():
             log("zip", "Render root 'Renders' not found; nothing to zip.")
         else:
             log("zip", "Archives already up to date; no sequences needed zipping.")
+        log(
+            "scan",
+            f"Summary: scanned {total_scanned}, quick-skipped {quick_skipped}, "
+            f"state-skipped {state_skipped}, empty {empty_dirs}, queued {queued}",
+        )
         return 0
 
     # Calculate RAM-aware worker count based on work items
-    worker_count, per_job_memory_limit = max_workers(requested_workers, work_items, verbose=verbose)
+    worker_count, ram_limits_dict = max_workers(requested_workers, work_items, verbose=verbose)
+    log(
+        "init",
+        f"Preparing to compress {len(work_items)} sequence(s) with {worker_count} worker(s)",
+    )
 
     updated_paths: list[Path] = []
 
@@ -738,7 +859,7 @@ def run_zip(requested_workers: int | None, *, verbose: bool) -> int:
 
     with ThreadPoolExecutor(max_workers=worker_count) as executor:
         future_map = {
-            executor.submit(process_zip, seq_dir, zip_path, state_path, seq_state, per_job_memory_limit, verbose=verbose): seq_dir
+            executor.submit(process_zip, seq_dir, zip_path, state_path, seq_state, ram_limits_dict.get(seq_dir), verbose=verbose): seq_dir
             for seq_dir, zip_path, state_path, seq_state in work_items
         }
 
@@ -758,6 +879,11 @@ def run_zip(requested_workers: int | None, *, verbose: bool) -> int:
             verbose_only=True,
             verbose=verbose,
         )
+    log(
+        "scan",
+        f"Summary: scanned {total_scanned}, quick-skipped {quick_skipped}, "
+        f"state-skipped {state_skipped}, empty {empty_dirs}, queued {queued}",
+    )
 
     removed = cleanup_orphan_archives(verbose=verbose)
     if removed:
@@ -851,6 +977,13 @@ def cleanup_orphan_archives(*, verbose: bool) -> int:
 
 def main() -> int:
     args = parse_args()
+    log("init", "zip_sequences starting up...")
+    log("init", f"Working directory: {Path.cwd()}")
+    log("init", f"Mode: {args.mode}, zipper: {ZIPPER_TYPE}, jobs arg: {args.jobs or 'auto'}")
+    if ZIPPER_TYPE == "7z":
+        exe = SEVEN_Z_EXE or "not found"
+        max_inst = MAX_7Z_INSTANCES if MAX_7Z_INSTANCES is not None else "auto"
+        log("init", f"7z executable: {exe}, Max7zInst: {max_inst}")
 
     if args.mode == "expand":
         # For expand mode, use simple CPU-based worker calculation