success at home

FLAMENCO_BUG_REPORT.md

@@ -0,0 +1,99 @@
+**System Information**
+Operating System(s): Win11 25h2
+
+**Flamenco Version**
+Is Broken:   3.8
+Worked OK: unknown
+
+When Flamenco is ran from a Windows machine, with the shared-storage set to an SMB share, the symlinks created by Shaman appear to be either completely useless at worst, context dependent at best.
+
+1. Download and configure Flamenco
+2. Configure the security policy as instructed [in the documentation](https://flamenco.blender.org/usage/shared-storage/shaman/#windows)
+3. Set `shaman: enabled: true`. 
+4. Set `shared_storage_path`. I was originally hosting the files on a TrueNAS SMB share (mapped to F:\). Testing it with a local drive (T:\Flamenco) results with the intended behavior.
+5. Submit job
+
+The symlinks created within `jobs` will not be universally accessible by Windows. All Windows workers (I have only tested Windows) will fail:
+
+```
+2026-01-08T15:51:58-07:00 Task assigned to worker i9kf (3690a968-f7ee-49d5-adb7-76154761d435)
+2026-01-08T15:51:58-07:00 task changed status queued -> active
+going to run:
+
+    'F:\software\Blender\blender.exe' -b -y --gpu-backend opengl 'F:\jobs/SSD_2B/SSD_2B.flamenco.blend' --python-expr '
+import bpy
+
+cycles_prefs = bpy.context.preferences.addons['"'"'cycles'"'"'].preferences
+cycles_prefs.compute_device_type = '"'"'OPTIX'"'"'
+for dev in cycles_prefs.get_devices_for_type('"'"'OPTIX'"'"'):
+    dev.use = (dev.type != '"'"'CPU'"'"')
+' --python-expr 'import bpy; bpy.context.scene.cycles.device = '"'"'GPU'"'"'' --render-output 'F:/renders/SSD_2B/2026-01-08_151057/######' --render-format PNG --render-frame 0..15
+
+pid=32124 > 00:10.047  reports          | ERROR File format is not supported in file "F:\jobs\SSD_2B\SSD_2B.flamenco.blend"
+pid=32124 > Blender 5.0.1 (hash a3db93c5b259 built 2025-12-16 01:32:30)
+pid=32124 > Cannot read ARP prefs
+pid=32124 > Update armature presets...
+pid=32124 > animation_layers_updater addon: Verbose is enabled
+pid=32124 > animation_layers_updater addon: Read in JSON settings from file
+pid=32124 > 
+pid=32124 > Blender quit
+Failed: command exited abnormally with code 1
+2026-01-08T15:52:11-07:00 Task failed by 1 worker, Manager will mark it as soft failure. 2 more failures will cause hard failure.
+2026-01-08T15:52:11-07:00 task changed status active -> soft-failed
+```
+
+Examining the jobfiles reveals the issue.
+<img width="480" alt="image.png" src="attachments/6e94046a-fdd3-439d-bcfc-7fbf33a449a7">
+The file is detected as `.symlink` type.
+<img width="304" alt="image.png" src="attachments/a11b8f53-916f-4437-ba8f-2b9ca09adda2">
+However, the symlink points to the proper Shaman Blob.
+
+If you right click and edit the symlink in notepad, it opens the blob:
+<img width="547" alt="image.png" src="attachments/0a23fbe9-4f26-40f2-a1da-215db6fb7ec0">
+A proper blendfile with gzip compression.
+
+When opened in VSCode, it opens blank.
+<img width="316" alt="image.png" src="attachments/0565d83f-ebda-4d84-99b5-e1ee5cc418c1">
+
+When copied to another folder, it copies a blank file.
+<img width="317" alt="image.png" src="attachments/003955ed-3e28-4307-8cc9-56b78ce13ac7">
+
+If you open the symlink in blender, the error is replicated:
+File format is not supported in file "F:\jobs\SSD_2B\SSD_2B.flamenco.blend"
+
+Opening the actual blob in blender works, but obviously it's still pointing to files that cannot be found from the blob's directory. I only did this to verify that the blobs were written correctly, and they clearly are.
+
+```
+18:03.360  blend            | Read blend: "F:\file-store\stored\14\0c6efa48408ce1ee0a7296e057e798b1a162d7bc60d542b03d8938f33776c8\1253809.blob"
+18:03.391  blend.readfile   | Read library: 'F:\file-store\stored\14\0c6efa48408ce1ee0a7296e057e798b1a162d7bc60d542b03d8938f33776c8\_outside_project\A\1 Amazon_Active_Projects\1 BlenderAssets\Amazon\Scenes\LivingRoom_v2.blend', '//_outside_project/A/1 Amazon_Active_Projects/1 BlenderAssets/Amazon/Scenes/LivingRoom_v2.blend', parent '<direct>'
+18:03.391  blend.readfile   | Cannot find lib 'F:\file-store\stored\14\0c6efa48408ce1ee0a7296e057e798b1a162d7bc60d542b03d8938f33776c8\_outside_project\A\1 Amazon_Active_Projects\1 BlenderAssets\Amazon\Scenes\LivingRoom_v2.blend'
+18:03.391  blend.readfile   | Read library: 'F:\file-store\stored\14\0c6efa48408ce1ee0a7296e057e798b1a162d7bc60d542b03d8938f33776c8\_outside_project\A\1 Amazon_Active_Projects\1 BlenderAssets\Amazon\Char\Customers\Mom.blend', '//_outside_project/A/1 Amazon_Active_Projects/1 BlenderAssets/Amazon/Char/Customers/Mom.blend', parent '<direct>'
+18:03.391  blend.readfile   | Cannot find lib 'F:\file-store\stored\14\0c6efa48408ce1ee0a7296e057e798b1a162d7bc60d542b03d8938f33776c8\_outside_project\A\1 Amazon_Active_Projects\1 BlenderAssets\Amazon\Char\Customers\Mom.blend'
+18:03.422  blend.readfile   | LIB: Collection: 'Living Room' missing from 'F:\file-store\stored\14\0c6efa48408ce1ee0a7296e057e798b1a162d7bc60d542b03d8938f33776c8\_outside_project\A\1 Amazon_Active_Projects\1 BlenderAssets\Amazon\Scenes\LivingRoom_v2.blend', parent '<direct>'
+18:03.422
+```
+
+### Root Cause
+
+Windows creates **Windows reparse points** (not Unix symlinks) when creating symlinks on SMB shares. The behavior differs based on how the file is accessed:
+
+1. **Windows reparse points on SMB shares:**
+   - Windows stores the target path in reparse point metadata
+   - Samba stores these as empty regular files on the server (doesn't understand Windows reparse points)
+   - The target path is correctly stored and can be read via `Get-Item` `.Target` property
+
+2. **Why some applications work:**
+   - **Windows Explorer/Notepad:** Use file system filters that detect reparse points and automatically resolve them client-side by reading the `Target` property and opening the target file
+   - This works because resolution happens on the Windows client, not the SMB server
+
+3. **Why Blender/other applications fail:**
+   - **Blender/VSCode/programmatic I/O:** Use direct file I/O operations (`ReadAllBytes()`, `FileStream`, etc.)
+   - These operations don't use reparse point filters
+   - SMB client doesn't automatically follow reparse points
+   - Returns the empty file from the server (0 bytes)
+
+## Workarounds
+
+1. **Use local storage:** Set `shared_storage_path` to a local NTFS drive (symlinks work perfectly)
+2. **Run Flamenco Manager on Linux/WSL:** Use `mfsymlinks` mount option for SMB shares
+3. Disable Shaman.

plan.md

@@ -0,0 +1,6 @@
+1. put windows and linux flamenco manager and worker on the shared storage (on linux machine that hosts the shared storage)
+2. set up two way variable for the shared storage path on both windows (F:\) and linux (/mnt/whatever/Flamenco)
+3. create batch to launch the manager from windows using ssh to the linux machine
+4. run batch to replace running the manager from windows
+
+i still need to submit a bug report that highlights that the symlinks are not working on windows when the shared storage is on an SMB share, which is the entire point... sybren does it on his demo video, so idk how it broke.

unified_flamenco_launcher.ps1

@@ -109,7 +109,7 @@ net use
 
 # Start worker
 Write-Host "Starting Flamenco worker..." -ForegroundColor Cyan
-Set-Location 'F:\software\Flamenco 3.7'
+Set-Location 'F:\software\Flamenco'
 if (Test-Path 'flamenco-worker.exe') {
     Write-Host "Running flamenco-worker.exe..." -ForegroundColor Green
     # Run the worker and capture its exit code
@@ -118,7 +118,7 @@ if (Test-Path 'flamenco-worker.exe') {
     Write-Host "Flamenco worker process has terminated with exit code: $exitCode" -ForegroundColor Yellow
     exit $exitCode  # Exit with the worker's exit code to trigger restart if needed
 } else {
-    Write-Host "Error: flamenco-worker.exe not found in F:\software\Flamenco 3.7" -ForegroundColor Red
+    Write-Host "Error: flamenco-worker.exe not found in F:\software\Flamenco" -ForegroundColor Red
     exit 1  # Exit with error code to trigger restart
 }
 '@
@@ -167,13 +167,13 @@ net use
 
 # Start worker via CMD - hardcoded paths 
 Write-Host "Running command file..." -ForegroundColor Cyan
-$defaultCmdPath = "F:\software\Flamenco 3.7\run-flamenco-worker.cmd"
+$defaultCmdPath = "F:\software\Flamenco\run-flamenco-worker.cmd"
 
 if (Test-Path $defaultCmdPath) {
-    Set-Location "F:\software\Flamenco 3.7"
+    Set-Location "F:\software\Flamenco"
     Write-Host "Starting worker..." -ForegroundColor Green
     # Use hardcoded path to avoid variable expansion issues
-    cmd.exe /c "F:\software\Flamenco 3.7\run-flamenco-worker.cmd"
+    cmd.exe /c "F:\software\Flamenco\run-flamenco-worker.cmd"
     Write-Host "Worker process has terminated." -ForegroundColor Yellow
 } else {
     Write-Host "Command file not found at default location." -ForegroundColor Red
@@ -236,7 +236,7 @@ net use
 
 # Simple direct command execution with automatic "2" input
 Write-Host "Running Flamenco worker..." -ForegroundColor Cyan
-Set-Location -Path "F:\software\Flamenco 3.7"
+Set-Location -Path "F:\software\Flamenco"
 if (Test-Path -Path "run-flamenco-worker.cmd") {
     # Create a temporary file to store the "2" input
     $tempInputFile = [System.IO.Path]::GetTempFileName()