Dropbox: clean up (remove) imported files

[ehrenfeu]

Dear all,

we're having the following scenario: OMERO 5.0.5 is running on a dedicated machine with the space for the binary repository being provided by a central storage machine via NFS. To use OMERO.dropbox, our server has an additional ~1TB of local storage attached, which is configured as the "watchDir" and being exposed to the users via Samba / CIFS.

From the thread auto-importer I'm reading that we could configure OMERO to remove files from the watchDir once the import has succeeded using

Code: Select all

--transfer=upload_rm

Is this correct? From how I understand, we can't use the "ln_rm" option as our watchDir and the ManagedRepository reside on independent file systems each, right?

Thanks
~Niko

[jmoore]

Hi Niko,

From the thread auto-importer I'm reading that we could configure OMERO to remove files from the watchDir once the import has succeeded using --transfer=upload_rm

Yup, that's just what it's intended for.

From how I understand, we can't use the "ln_rm" option as our watchDir and the ManagedRepository reside on independent file systems each, right?

Also correct: regular hard-linking boundaries apply. And a "ln_s_rm" wouldn't get you very much! (That being said, if you'd like to try out a "cp_rm" we could certainly give it a try.)

Cheers,
~Josh.

[ehrenfeu]

Hi Josh,

thanks for the update! Are we correct in assuming this option has to go here?

Code: Select all

<property name="omero.fs.importArgs"  value=""/>

Cheers
~Niko

[cblackburn]

Hi Niko,

Yes, that's correct. You can also set the configuration via the command line, as illustrated here,

https://www.openmicroscopy.org/site/sup ... ml#dropbox

Cheers,

Colin

[ehrenfeu]

Thanks for the info! We'll try and see how it goes...

Josh, could you tell us details what's different in "cp_rm"?

Thanks
~Niko

[jmoore]

Hi Niko,

`cp_rm` doesn't exist yet, but it's a pretty simple manner of modifying this line in HardlinkFileTransfer to use "cp" rather than "ln". The difference is that a full copy takes place but locally and likely much faster than uploading via Ice (though I haven't compared the speed yet).

Cheers,
~Josh.

[jmoore]

Hi Niko,

just a follow-up that https://github.com/openmicroscopy/openm ... /pull/3379 has been merged so `cp_rm` should be in the nightly builds as well as 5.0.7 for your review.

All the best,
~Josh.