Raincloud/sheepit-shadow-nabber
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Feature: support for shared downloads directory (#295)

Browse Source 
* Feature: support for shared downloads directory

This feature is especially relevant for users with several clients running simultaneously within the same computer (computers running several GPUs or any combination of GPUs + CPUs) or multiple computers running in a network. The feature allows the user to specify a shared downloads directory via the new -shared-downloads-dir client option. All the clients using that option will share the same binaries and scene files.

How it works?
The first client downloading a binary or scene will save the file in the directory especified in -shared-downloads-directory. The rest of the clients using the same option will wait until the file has been downloaded. Once the file has been downloaded, it will be ready for the rest of the clients.

This feature is especially relevant for users with metered/slow connections or multiple computers/clients that don't want to download the same binary/file multiple times.

IMPORTANT: All the clients intended to share the binaries and scenes must execute the client with the same -shared-downloads-dir parameter.
This commit is contained in:
Luis Uguina
2020-10-21 22:03:09 +11:00
committed by GitHub
parent 70d7dc052c
commit 6587b4dcef
5 changed files with 157 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
src/com/sheepit/client/Server.java
View File

@@ -433,7 +433,7 @@ public class Server extends Thread {
}
is = response.body().byteStream();
output = new FileOutputStream(destination_);
output = new FileOutputStream(destination_ + ".partial");
long size = response.body().contentLength();
byte[] buffer = new byte[8 * 1024];
@@ -488,6 +488,17 @@ public class Server extends Thread {
output.close();
}
File downloadedFile = new File(destination_ + ".partial");
if (downloadedFile.exists()) {
// Rename file (or directory)
boolean success = downloadedFile.renameTo(new File(destination_));
if (!success) {
this.log.debug(String.format("Server::HTTPGetFile Error trying to rename the downloaded file to final name (%s)", destination_));
}
}
if (is != null) {
is.close();
}
@@ -629,6 +640,14 @@ public class Server extends Thread {
File file_to_delete = new File(path + ".zip");
file_to_delete.delete();
Utils.delete(new File(path));
// If we are using a shared downloads directory, then delete the file from the shared downloads directory as well :)
if (this.user_config.getSharedDownloadsDirectory() != null) {
String commonCacheFile = this.user_config.getSharedDownloadsDirectory().getAbsolutePath() + File.separatorChar + fileMD5.getMd5();
this.log.debug("Server::handleFileMD5DeleteDocument delete common file " + commonCacheFile + ".zip");
file_to_delete = new File(commonCacheFile + ".zip");
file_to_delete.delete();
}
}
}
}