Batch Image Export

[mattc]

Hi,

I have a couple of questions about the batch image export script. When I export using everything set to the defaults it seems to work but i'm not sure where to find the Batch_Image_Export directory that is presumably created. When I try and export as OME-TIFF it fails with the error:

{'Export_Individual_Channels': True, 'Choose_Z_Section': 'Default-Z (last-viewed)', 'Choose_T_Section': 'Default-T (last-viewed)', 'Data_Type': 'Image', 'Format': 'OME-TIFF', 'IDs': [19726L, 19719L, 19725L], 'Zoom': '100%', 'Export_Merged_Image': True, 'Folder_Name': 'Batch_Image_Export', 'Individual_Channels_Grey': False}
Processing 3 images

WARNING:omero.gateway:LockTimeout on <class 'omero.gateway.OmeroGatewaySafeCallWrapper'> to <7e436148-2788-45a7-ac76-e2e930785139omero.api.RenderingEngine> load((<ServiceOptsDict: {'omero.session.uuid': 'd0c132fc-5a27-4a79-8366-ac8b64321067', 'omero.group': '53', 'omero.client.uuid': '903c2f76-4613-4ca1-9ea1-ea828a8c5cfb'}>,), {})
Traceback (most recent call last):
File "/home/omero/OMERO.server-5.0.6-ice35-b53/lib/python/omero/gateway/__init__.py", line 3635, in __call__
return self.f(*args, **kwargs)
File "/home/omero/OMERO.server-5.0.6-ice35-b53/lib/python/omero_api_RenderingEngine_ice.py", line 416, in load
return _M_omero.api.RenderingEngine._op_load.invoke(self, ((), _ctx))
LockTimeout: exception ::omero::LockTimeout
{
...
}

Any help much appreciated!

Thanks,

Matt.

[wmoore]

Hi Matt,

The Batch Image Export.zip file will be attached to one of the images you chose to export. If you're running the script from web, the Activities panel will show you a 'folder icon' button that will browse you to the image that is annotated with the File attachment (file attachments shown in right panel). Insight does the same with a "Go to Attachment" menu option (when you click on the zip name in the Activities panel).

For you OME-TIFF export - can you tell us how big the image is you're trying to export? (size X, Y & Z). Does the same image(s) work for export of other formats (e.g. default options)?

An alternative strategy is to use Insight's "Export as OME-TIFF" option under the "Download" button top of the right panel when a single image is selected.

Regards,

Will.

[mattc]

Thanks Will, the export seems to be working now. I think our server may have been acting up when I ran the test. Now that the export works correctly I can see the Batch Image Export.zip button so that solves that too. Thanks again for your help.

[mattc]

Hi again,

I'm encountering what looks like a slightly different problem with the batch image export. I'm using the OMERO plugin to Fiji and i'm trying to export an image from my own user area. It works OK as a JPEG or a TIFF but when I try as an OME-TIFF I get an error saying that the file is too large to export:

WARNING:omero.gateway:InternalException on <class 'omero.gateway.OmeroGatewaySafeCallWrapper'> to <7a58b14e-f271-40a1-9d56-dc3be3416f76omero.api.Exporter> generateTiff((<ServiceOptsDict: {'omero.session.uuid': '3c57fbcb-d0cf-46f2-acf0-b23de0625776', 'omero.client.uuid': '5bd2cc7f-d77b-4021-86b8-782e4dab700e'}>,), {})
Traceback (most recent call last):
File "/home/omero/OMERO.server-5.2.0-ice35-b12/lib/python/omero/gateway/__init__.py", line 4202, in __call__
return self.f(*args, **kwargs)
File "/home/omero/OMERO.server-5.2.0-ice35-b12/lib/python/omero_api_Exporter_ice.py", line 170, in generateTiff
return _M_omero.api.Exporter._op_generateTiff.invoke(self, ((), _ctx))
InternalException: exception ::omero::InternalException
{
serverStackTrace = omero.ApiUsageException
serverStackTrace = (null)
serverExceptionClass = (null)
message = "Image:253 is too large for export (sizeX=4000, sizeY=2988)"
at ome.services.blitz.impl.ExporterI$2.doWork(ExporterI.java:365)
at sun.reflect.GeneratedMethodAccessor311.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at ome.services.util.Executor$Impl$Interceptor.invoke(Executor.java:562)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at ome.security.basic.EventHandler.invoke(EventHandler.java:154)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:111)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy72.doWork(Unknown Source)
at ome.services.util.Executor$Impl.execute(Executor.java:443)
at ome.services.util.Executor$Impl.execute(Executor.java:387)
at ome.services.blitz.impl.ExporterI.do_tiff(ExporterI.java:338)
at ome.services.blitz.impl.ExporterI.generateTiff_async(ExporterI.java:204)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at omero.cmd.CallContext.invoke(CallContext.java:78)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy118.generateTiff_async(Unknown Source)
at omero.api._ExporterTie.generateTiff_async(_ExporterTie.java:72)
at omero.api._ExporterDisp.___generateTiff(_ExporterDisp.java:262)
at omero.api._ExporterDisp.__dispatch(_ExporterDisp.java:332)
at IceInternal.Incoming.invoke(Incoming.java:222)
at Ice.ConnectionI.invokeAll(ConnectionI.java:2482)
at Ice.ConnectionI.dispatch(ConnectionI.java:1258)
at Ice.ConnectionI.message(ConnectionI.java:1213)
at IceInternal.ThreadPool.run(ThreadPool.java:321)
at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:693)
at java.lang.Thread.run(Thread.java:745)

serverExceptionClass = omero.ApiUsageException
message =
}
Traceback (most recent call last):
File "./script", line 626, in <module>
runScript()
File "./script", line 611, in runScript
fileAnnotation, message = batchImageExport(conn, scriptParams)
File "./script", line 410, in batchImageExport
saveAsOmeTiff(conn, img, folder_name)
File "./script", line 195, in saveAsOmeTiff
fileSize, block_gen = image.exportOmeTiff(bufsize=65536)
File "/home/omero/OMERO.server-5.2.0-ice35-b12/lib/python/omero/gateway/__init__.py", line 8096, in exportOmeTiff
size = e.generateTiff(self._conn.SERVICE_OPTS)
File "/home/omero/OMERO.server-5.2.0-ice35-b12/lib/python/omero/gateway/__init__.py", line 4205, in __call__
return self.handle_exception(e, *args, **kwargs)
File "/home/omero/OMERO.server-5.2.0-ice35-b12/lib/python/omero/gateway/__init__.py", line 4202, in __call__
return self.f(*args, **kwargs)
File "/home/omero/OMERO.server-5.2.0-ice35-b12/lib/python/omero_api_Exporter_ice.py", line 170, in generateTiff
return _M_omero.api.Exporter._op_generateTiff.invoke(self, ((), _ctx))
omero.InternalException: exception ::omero::InternalException
{
serverStackTrace = omero.ApiUsageException
serverStackTrace = (null)
serverExceptionClass = (null)
message = "Image:253 is too large for export (sizeX=4000, sizeY=2988)"
at ome.services.blitz.impl.ExporterI$2.doWork(ExporterI.java:365)
at sun.reflect.GeneratedMethodAccessor311.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at ome.services.util.Executor$Impl$Interceptor.invoke(Executor.java:562)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at ome.security.basic.EventHandler.invoke(EventHandler.java:154)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:111)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy72.doWork(Unknown Source)
at ome.services.util.Executor$Impl.execute(Executor.java:443)
at ome.services.util.Executor$Impl.execute(Executor.java:387)
at ome.services.blitz.impl.ExporterI.do_tiff(ExporterI.java:338)
at ome.services.blitz.impl.ExporterI.generateTiff_async(ExporterI.java:204)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at omero.cmd.CallContext.invoke(CallContext.java:78)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy118.generateTiff_async(Unknown Source)
at omero.api._ExporterTie.generateTiff_async(_ExporterTie.java:72)
at omero.api._ExporterDisp.___generateTiff(_ExporterDisp.java:262)
at omero.api._ExporterDisp.__dispatch(_ExporterDisp.java:332)
at IceInternal.Incoming.invoke(Incoming.java:222)
at Ice.ConnectionI.invokeAll(ConnectionI.java:2482)
at Ice.ConnectionI.dispatch(ConnectionI.java:1258)
at Ice.ConnectionI.message(ConnectionI.java:1213)
at IceInternal.ThreadPool.run(ThreadPool.java:321)
at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:693)
at java.lang.Thread.run(Thread.java:745)

serverExceptionClass = omero.ApiUsageException
message =
}

Is there a configurable limit on the size of files to be exported?

Thanks for your help,

Matt.

[jmoore]

Is there a configurable limit on the size of files to be exported?

It's a server-wider configuration. The requiresPyramid method used consults the configuration properties omero.pixeldata.max_plane_height and omero.pixeldata.max_plane_width, by default 3192x3192. If you have sufficient memory, you can raise the value, but there are both practical and theoretical limits. Supporting a tiled-export is certainly on our priority list.

Cheers,
~Josh.