Prev/next page links do not adapt to filtered results

[Sethur]

Hi,

We just updated to OMERO 5.0.3. and the following issue came up (possible also present in previous version):

When using OMERO.web to filter a large images (by typing some filter words into the box at the top of the middle area), the Prev/next page links at the bottom of the page do not adapt to the reduced amount of available images. When someone, for example, filters with something that matches only a single image, the Prev/next page links stay exactly the same although they should disappear.

Cheers,

Tristan

[manics]

Thanks Tristan. I've opened a ticket and added you to the CC list.

Simon

[Sethur]

Hi Simon,

thanks for opening the ticket. You stated that the fact that filtering only applies to the current page would make fixing this bug difficult.

Now that you mention this: A while ago I realized that the filtering seemingly only applies to the current page and not to the selected dataset or project. I wanted to open a thread about it but, apparently, I forgot since I normally don't work with large datasets. This behavior of OMERO.web is counterintuitive for people who are using OMERO for the first time as well as inefficient for users having datasets with a lot of images. If a user has to find the page containing the images he wants his filter to apply to before he can apply his filter, he may just as well not use the feature at all.

Is there a reason for not fixing this and thus also fixing the issue with the page links while applying a filter?

Cheers,

Tristan

[wmoore]

Hi Tristan,

The filter feature is purely a Javascript filtering of data that you already have in the browser and it doesn't load any data from the server. This means that it's pretty fast (happens while you type) but it's limited to the images you've already loaded (doesn't work well with pagination).

So it's quite a big change to make this work with pages.
The filtering would have to happen on the server, and probably wouldn't be fast enough to happen while you type.

[Sethur]

Hi Simon,

How about loading the complete list of bandwith-wise inexpensive infos (i.e. no thumbnails, but name, location, etc.) from the server regardless of the dataset size and controlling the displaying of thumbnails in a dynamic matter (i.e. if you scroll down, you will see more thumbnails appearing).

That way you would have all the names readily available for as-you-type javascript filtering and if the filtered list contains new thumbnails they would be loaded on demand.

Another option would be doing the filtering on the server and abandoning the as-you-type functionality as soons as there is more than one page.

In my opinion, filtering only the images visible in one page is not a viable long-term solution. We already had confused users searching for images uploaded via DropBox and not finding them because they thought the filter would apply to everything inside Orphaned Images.

Cheers,

Tristan

[PaulVanSchayck]

Dear Josh,

I also agree with Tristan, that loading several thousands rows in JavaScript should be no problem. Take a look at for example DataTables which does client side filtering on tables in Javascript.
However, in that case the current method of DOM filtering should be rewritten, as storing data in the DOM would not be suitable for larger numbers. Storing the data in pure javascripts arrays and objects would be suitable for tens of thousands of images.
Of course, once a dataset goes into the millions of images javascript becomes unsuitable again and server side filtering is required.

Kind regards,

Paul

[wmoore]

Thanks both for your suggestions.

I agree that it would be nice to have the features you're asking for.
We'll add it to the TODO list and start scoping out the work...

Regards,

Will.

[Sethur]

Hi Will, hi Josh,

a minute ago another frustrated user who was unable to find his images inside the DropBox (orphaned images) folder left my office. I had to explain to him that the filtering feature works as discussed above to which he commented that nobody would expect it to work like that (something I totally agree to). I could see that he was very displeased on having wasted so much time on such a simple task.

In order to avoid future frustration with this particular user, I wanted to help him on the spot by cleaning out his "orphaned images" folder and creating some datasets for him. Alas, this was not possible (via OMERO.web) since even administrators cannot create projects or datasets for other users. Again, this is something that an IT-experienced person would not expect (most systems implement such a feature for superusers) and it made administrating the system unpleasent in more than one occation. I meantioned this issue in a previous post of mine http://www.openmicroscopy.org/community/viewtopic.php?f=4&t=7484 but I don't think it reached I high position in the priority list.

So what I want to say is this: From my point of view, the fact that the filter is not working on all pages is not just a missing feature. It's a bug that makes users want to not use OMERO any longer. I would suggest to fix this with the next release. The issue with the projects and datasets is less critical (and actually belongs in a separate thread), but there should be a fix for this, too, in the very near future. Admins need to have full controll over the data in the system without the need to employ commands on the CLI or manually messing around with the database.

Cheers,

Tris

[manics]

Hi Tristan

What's the typical size of your datasets or Orphaned images folder? I'm just wondering if increasing the number of images on a page to a sufficiently large number could work as a temporary fix.

Simon

[Sethur]

Hi Simon,

unfortunately the typical size varies a lot since users tend to be to lazy to sort the images they put on the DropBox. Most of them just want to download the images later on to continue with their analysis steps as they had when they still copied everything on USB sticks.

I agree, though, that increasing this limit to around 200 would at least temporary solve this issue for most users at the cost of increased server load and bandwidth. Can you point me to the respective piece of code in OMERO.web?

Cheers,

Tristan