Raincloud/ProjectStructure_HOME
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
dfc113ea38501b4dbe9ba5a7ed45b4d15e93bccf
ProjectStructure_HOME/.specstory/history/2025-11-10_16-59Z-change-compression-method-to-7z.md
Nathan 5c5e22cfb7 change config bools
2025-11-10 10:24:55 -07:00

15 KiB
Raw Blame History

Change compression method to 7z (2025-11-10 16:59Z)

User (2025-11-10 16:59Z)

@zip_sequences.py (1-334) let's have it use 7z at compression lvl 9 instead of zip at 0 (store) I don't mind it taking a few minutes per zip if they're more compressed.

Agent (model default, mode Agent)

Switching to 7z with compression level 9. Updating the code to use 7z instead of Python's zipfile.

Tool use: **read_file** • Read file: zip_sequences.py

Updating the code to use 7z with compression level 9 instead of Python's zipfile with no compression.

Tool use: **code_edit** • Edit file: zip_sequences.py

Chunk 1 Lines added: 1, lines removed: undefined

@@ -14,4 +14,5 @@
  import os
  import shutil
+ import subprocess
  import sys
  from concurrent.futures import ThreadPoolExecutor, as_completed
Tool use: **code_edit** • Edit file: zip_sequences.py

Chunk 1 Lines added: 1, lines removed: 1

@@ -127,5 +127,5 @@
  def archive_path_for(seq_dir: Path) -> Path:
      rel = seq_dir.relative_to(RENDER_ROOT)
-     return (ARCHIVE_ROOT / rel).with_suffix(".zip")
+     return (ARCHIVE_ROOT / rel).with_suffix(".7z")
Tool use: **code_edit** • Edit file: zip_sequences.py

Chunk 1 Lines added: 30, lines removed: 5

@@ -140,10 +140,35 @@
  
  def zip_sequence(seq_dir: Path, zip_path: Path) -> None:
-     from zipfile import ZIP_STORED, ZipFile
- 
      zip_path.parent.mkdir(parents=True, exist_ok=True)
-     with ZipFile(zip_path, "w", compression=ZIP_STORED) as archive:
-         for file_path in iter_sequence_files(seq_dir):
-             archive.write(file_path, arcname=file_path.relative_to(seq_dir).as_posix())
+     
+     # Build list of files to archive with relative paths
+     file_list = []
+     for file_path in iter_sequence_files(seq_dir):
+         rel_path = file_path.relative_to(seq_dir).as_posix()
+         file_list.append(rel_path)
+     
+     # Use 7z with maximum compression (level 9)
+     # -mx=9: maximum compression
+     # -mmt=on: multi-threading
+     # -y: assume yes to all prompts
+     cmd = [
+         "7z", "a",
+         "-mx=9",
+         "-mmt=on",
+         "-y",
+         str(zip_path),
+     ]
+     
+     # Change to sequence directory and add files
+     result = subprocess.run(
+         cmd + file_list,
+         cwd=seq_dir,
+         capture_output=True,
+         text=True,
+         check=False,
+     )
+     
+     if result.returncode != 0:
+         raise RuntimeError(f"7z compression failed: {result.stderr}")
Tool use: **code_edit** • Edit file: zip_sequences.py

Chunk 1 Lines added: 19, lines removed: 4

@@ -174,13 +174,28 @@
  
  def expand_sequence(zip_path: Path, seq_state: dict) -> None:
-     from zipfile import ZipFile
- 
      target_dir = sequence_dir_for(zip_path)
      if target_dir.exists():
          shutil.rmtree(target_dir)
      target_dir.mkdir(parents=True, exist_ok=True)
  
-     with ZipFile(zip_path, "r") as archive:
-         archive.extractall(target_dir)
+     # Use 7z to extract archive
+     # -y: assume yes to all prompts
+     # -o: output directory
+     cmd = [
+         "7z", "x",
+         "-y",
+         f"-o{target_dir}",
+         str(zip_path),
+     ]
+     
+     result = subprocess.run(
+         cmd,
+         capture_output=True,
+         text=True,
+         check=False,
+     )
+     
+     if result.returncode != 0:
+         raise RuntimeError(f"7z extraction failed: {result.stderr}")
  
      for entry in seq_state.get("files", []):
Tool use: **code_edit** • Edit file: zip_sequences.py

Chunk 1 Lines added: 1, lines removed: 1

@@ -284,5 +284,5 @@
      work_items: list[tuple[Path, dict]] = []
  
-     for zip_path in ARCHIVE_ROOT.rglob("*.zip"):
+     for zip_path in ARCHIVE_ROOT.rglob("*.7z"):
          state_path = state_path_for(zip_path)
          seq_state = load_state(state_path)

Chunk 2 Lines added: 1, lines removed: 1

@@ -327,5 +327,5 @@
      removed: list[Path] = []
  
-     for zip_path in ARCHIVE_ROOT.rglob("*.zip"):
+     for zip_path in ARCHIVE_ROOT.rglob("*.7z"):
          seq_dir = sequence_dir_for(zip_path)
          if seq_dir.exists():

Updating the docstring to reflect 7z usage and checking for lint errors:

Tool use: **code_edit** • Edit file: zip_sequences.py

Chunk 1 Lines added: 5, lines removed: 5

@@ -1,9 +1,9 @@
  #!/usr/bin/env python3
- """Maintain zipped render sequences for Git hooks.
+ """Maintain compressed render sequences for Git hooks using 7z.
  
- Default mode scans `Renders/`, produces ZIP archives under `Renders/_zipped/`,
- and stages any updated archives so commits only track compact files. Switch to
- `--mode expand` to inflate the tracked archives back into the ignored working
- directories after checkouts or pulls.
+ Default mode scans `Renders/`, produces 7z archives (compression level 9) under
+ `Renders/_zipped/`, and stages any updated archives so commits only track compact
+ files. Switch to `--mode expand` to inflate the tracked archives back into the
+ ignored working directories after checkouts or pulls.
  """
Tool use: **read_lints** • Read lints for 1 file

Lint paths:

  • zip_sequences.py

Lint results

No lint errors found

Updated the script to use 7z with compression level 9 instead of Python's zipfile with no compression. Changes:

  1. Added subprocess import for calling 7z
  2. Changed archive format from .zip to .7z
  3. Replaced zip_sequence function to use 7z a with:
    • -mx=9: maximum compression
    • -mmt=on: multi-threading
    • -y: assume yes to prompts
  4. Replaced expand_sequence function to use 7z x for extraction
  5. Updated all file pattern searches from *.zip to *.7z
  6. Updated docstring to reflect 7z usage

The script now creates 7z archives with maximum compression. Compression may take longer, but archives will be smaller. Ensure 7z is installed and available in your PATH (typically 7z.exe on Windows).

User (2025-11-10 16:59Z)

Will this remove old zip versions, or will I have to delete the _zipped folder in repo and re-compress?

Agent (model default, mode Agent)

Checking the cleanup logic to see if it handles old .zip files:

Tool use: **read_file** • Read file: zip_sequences.py
Tool use: **read_file** • Read file: zip_sequences.py

The cleanup function only looks for .7z files, so old .zip files won't be removed. Updating it to also remove obsolete .zip files:

Tool use: **code_edit** • Edit file: zip_sequences.py

Chunk 1 Lines added: 11, lines removed: undefined

@@ -327,4 +327,15 @@
      removed: list[Path] = []
  
+     # Remove obsolete .zip files (migrated to .7z)
+     for old_zip in ARCHIVE_ROOT.rglob("*.zip"):
+         rel = old_zip.relative_to(ARCHIVE_ROOT)
+         log("zip", f"Removing obsolete .zip archive {rel}", verbose_only=True, verbose=verbose)
+         old_zip.unlink(missing_ok=True)
+         state_path = state_path_for(old_zip)
+         if state_path.exists():
+             state_path.unlink()
+         removed.append(old_zip)
+ 
+     # Remove orphan .7z archives (sequence directory no longer exists)
      for zip_path in ARCHIVE_ROOT.rglob("*.7z"):
          seq_dir = sequence_dir_for(zip_path)
Tool use: **read_lints** • Read lints for 1 file

Lint paths:

  • zip_sequences.py

Lint results

No lint errors found

Updated the cleanup function to automatically remove old .zip files. When you run the script, it will:

  1. Remove obsolete .zip files (migrated to .7z)
  2. Remove orphan .7z archives (when the sequence directory no longer exists)
  3. Clean up associated .meta.json state files

No need to manually delete the _zipped folder. The next run will remove old .zip files and create new .7z archives with maximum compression. The cleanup runs automatically after the zip operation completes.

User (2025-11-10 16:59Z)

I just pulled from HOME. Let's change DailyFormat from a true/false bool to a string so I can just put the format in (daily_YYMMDD vs YYYY-MM-DD) and the same thing for zipper, "7z" or "zip"

Agent (model default, mode Agent)

Checking config.json, then updating the code to use string values for DailyFormat and zipper.

Tool use: **read_file** • Read file: config.json
Tool use: **read_file** • Read file: zip_sequences.py

Searching for dailyFormat usage in the file:

Tool use: **grep** • Grep for "dailyFormat|daily_format|DailyFormat" in "zip_sequences.py" • 0 matches

Output mode: content

No matches found

Updating the code to use string values for both settings:

Tool use: **code_edit** • Edit file: zip_sequences.py

Chunk 1 Lines added: 2, lines removed: 1

@@ -34,6 +34,7 @@
  CONFIG_PATH = Path(__file__).resolve().with_name("config.json")
  DEFAULT_CONFIG = {
-     "zipper": True,
+     "zipper": "7z",
      "compression": 9,
+     "dailyFormat": "daily_YYMMDD",
  }
Tool use: **code_edit** • Edit file: zip_sequences.py

Chunk 1 Lines added: 2, lines removed: 1

@@ -62,5 +62,6 @@
  
  CONFIG = load_config()
- USE_7Z = bool(CONFIG.get("zipper", True))
+ ZIPPER_TYPE = str(CONFIG.get("zipper", "7z")).lower()
+ USE_7Z = ZIPPER_TYPE == "7z"
  COMPRESSION_LEVEL = CONFIG.get("compression", 9)
  if isinstance(COMPRESSION_LEVEL, str):
Tool use: **read_lints** • Read lints for 1 file

Lint paths:

  • zip_sequences.py

Lint results

No lint errors found

Adding backward compatibility for boolean values in the config:

Tool use: **code_edit** • Edit file: zip_sequences.py

Chunk 1 Lines added: 6, lines removed: 1

@@ -62,5 +62,10 @@
  
  CONFIG = load_config()
- ZIPPER_TYPE = str(CONFIG.get("zipper", "7z")).lower()
+ zipper_val = CONFIG.get("zipper", "7z")
+ # Handle both old boolean format and new string format
+ if isinstance(zipper_val, bool):
+     ZIPPER_TYPE = "7z" if zipper_val else "zip"
+ else:
+     ZIPPER_TYPE = str(zipper_val).lower()
  USE_7Z = ZIPPER_TYPE == "7z"
  COMPRESSION_LEVEL = CONFIG.get("compression", 9)
Tool use: **read_lints** • Read lints for 1 file

Lint paths:

  • zip_sequences.py

Lint results

No lint errors found

Updating config.json to use the new string format:

Tool use: **read_file** • Read file: config.json
Tool use: **code_edit** • Edit file: config.json

Chunk 1 Lines added: 2, lines removed: 2

@@ -1,6 +1,6 @@
  {
-   "dailyFormat": true,
+   "dailyFormat": "daily_YYMMDD",
    "structDir": "D:\\ProjectStructure",
-   "zipper": true,
+   "zipper": "7z",
    "compression": 9
  }

Updated both files:

  1. zip_sequences.py:

    • Changed DEFAULT_CONFIG to use string values: "zipper": "7z" and "dailyFormat": "daily_YYMMDD"
    • Updated the zipper logic to accept string values ("7z" or "zip") with backward compatibility for boolean values
    • The code now checks if zipper is "7z" (case-insensitive) to determine whether to use 7z compression

  2. config.json:

    • Updated "zipper": true → "zipper": "7z"
    • Updated "dailyFormat": true → "dailyFormat": "daily_YYMMDD"

You can now set:

  • "zipper": "7z" or "zipper": "zip" to choose the compression tool
  • "dailyFormat": "daily_YYMMDD" or "dailyFormat": "YYYY-MM-DD" or any other format string

The code still supports old boolean values for backward compatibility, so existing configs will continue to work.

Reference in New Issue View Git Blame Copy Permalink