download original Files takes much time

[saleht]

Hi all,

during trying to trace the cause of the issue http://www.openmicroscopy.org/community ... f=4&t=8404

download original files takes really much time
steps:
- version 5.4.1
1. sigin to Omero using Omero web
2. select 5 images (size around 600 MB)
3. trigger download
- issue: Preparing zip for download takes for 600 Mb(this size after zipping ) about 7 min 25 sec which i believe is too much

i think the issue happens because of zipping large files, and that takes time. IS there a way to download all of them without zipping them (like Omero.insight) ?
any suggestions ?

[wmoore]

Hi,

Unfortunately it just takes time to download all the files into a temp directory on the web server then build a zip from them, although 7 minutes does seem a little long.

If the number of files is not too large, you can simply click on each individual file in the download dialog (see screenshot) to download each one directly from the OMERO server. This should be much more efficient (download should start immediately) since we don't need to store the files on the web server.

If the number of files is too high, it may be possible to trigger the download of all files using some JavaScript in the developer tools Console to open a new tab for each download link:
e.g.

Code: Select all

$("table a").each(function(){window.open($(this).attr('href'), "_blank")})


but this may be blocked by your browser popup blocker and it could also place a heavy load on your server so should be tried with caution.

Regards,

Will.

[saleht]

Hi will,
thx for your answer

Unfortunately it just takes time to download all the files into a temp directory on the web server then build a zip from them, although 7 minutes does seem a little long.

yeah i noticed that, when i try to download my tmp will be very big,
the question is, can i change the place of tmp folder, i want to move it to bigger partition, becasue root partition has 10 Gb free and this this critical.

[wmoore]

Hi,

OMERO.web uses python's tempfile module for creating the temp directory and zip file.
You can specify where this is by setting various environment variables as described: https://docs.python.org/2/library/tempf ... le.tempdir

Regards,

Will.

[saleht]

Hi will,
i did the following to change the tmp dir
import tempfile
tempfile.mktemp(dir='/../..')
and then checking the python tmp dir
tempfile.gettempdir()
i am getting the new dirctory
but after a while it will come back to old or Default one
i am not sure if what i did right

[jmoore]

Hi Saleh,

what do you mean, "after a while it will come back"? After you restart the server(s)? Something else?

Cheers,
~Josh

[saleht]

Hi Josh,

No even without a restart, maybe after 3 or 5 mins, will changed again Automatically
Cheers, Saleh

[jmoore]

Hi Saleh,

that sounds quite surprising. Can you try outlining your steps here for me, since I'm coming to this thread late?

We're talking about one server that you're running in production, right? Or do you have a test server?

How are you setting tempdir? What value are you using?

Try to give us as much detail as possible.

All the best,
~Josh.

[saleht]

Hi Josh,

it is actually a test server, but i need to apply this on my production one.
steps:
1. open python terminal by writing python on Linux's terminal
2. import tempfile
3. tempfile.mkdtmp(dir='/data/pythonTmp')
4. tempfile.gettmpdir()


4 th step is to check the tmp directory

i am not sure if these steps is correct
cheers, Saleh

[jmoore]

Hi Saleh,

I understand. `tempfile.mkdtemp` does not change the return value of `tempfile.gettmpdir()` but rather returns you a new directory:

Code: Select all

cd /tmp
mkdir -p josh
python -c "import tempfile; print tempfile.mkdtemp(dir='/tmp/josh'); print tempfile.gettempdir()"


prints out:

Code: Select all

/tmp/josh/tmp7_yDQs
/tmp


whereas if I change my environment variable:

Code: Select all

cd /tmp
mkdir -p josh

export TMPDIR=/tmp/josh
python -c "import tempfile; print tempfile.mkdtemp(dir='/tmp/josh'); print tempfile.gettempdir()"


I see:

Code: Select all

/tmp/josh/tmpnK9vpy
/tmp/josh


Cheers,
~Josh