Deleting images doesn't free space

[dgarneau]

Hi,

I recently deleted many images to free some space on our server but found out, after the images were removed, that the space doesn't seems to be freed. I didn't delete the images directly but the project (or the dataset I don't remember) containing the images. Can it be the problem ?

Can the images still be there or is it a problem with storage calculation ? If the images are still there how can I delete them ?

As additional information, I looked at the space using the Admin tool in Omero web and I deleted the images with Insight. We are using OMERO 5.2.2. In insight if I go in the "Image" tab (in the left) it mention there is no images for that user.

Thank you
Daniel

[wmoore]

Hi Daniel,

When you delete a Dataset, this should delete the images it contains *unless* the images are also in another Dataset.

For the images you've already deleted, you could try to search for them (in Insight or web client) to see if you can find them in another Dataset.

If you're trying this again with other images, before you delete the Dataset, you can tell if the images are in other Datasets by selecting an image and in the right panel look under "Located in" pane in Insight (bottom of General tab) or under the 'tree' icon at top toolbar in right panel of webclient.
Also, if you click the 'link' button (top of right panel in webclient) you can get a bookmark link to an image. Then you can check after deleting whether the image has gone by trying to visit that link in your browser.

You can directly delete the images (select the images and 'delete') which will delete them even if they are in multiple Datasets.

How did you import these images?
If the are an "in place" import via the command line, then their location on disk will not affect the disk usage in OMERO so you will not see a change when they are deleted.

If you see no images for that user in the Images tab in Insight, that would suggest that they have been deleted (if you saw them there before deleting?).

Hope some of this is useful. Let us know if this doesn't explain what you're seeing.

Cheers,

Will.

[dgarneau]

Hi,

I cannot find these images anywhere else. What's weird is, if I look under the group and user in the Admin tool in OMERO web, it tells me the user is using 168 GB of space but if I browse for images under this group/user, I cannot see any images. One other thing, if I go to History in OMERO web, I can still see some folder icons but when I click on them I got a message "No data".

the images were imported using Insight.
Is there a way I can clear that space ?

Thank you again for your help,
Daniel

[manics]

Hi Daniel

If you've got access to the omero command line could you try the following, to list any additional files that might have been uploaded?

Code: Select all

$ omero -u <username> -g <groupname> --sudo root hql 'FROM OriginalFile'
Previous session expired for <username> on xxx:4064
Server: [eel:4064]
Password for root:
Created session XXXXX (<username>@xxx:4064). Idle timeout: 10 min. Current group: private-1
...


See https://www.openmicroscopy.org/site/sup ... sions.html if you want more explanation about the command line parameters. If you also run this with Image instead of OriginalFile you should see whether there are additional images that Insight isn't seeing for whatever reason.

[dgarneau]

Hi,

We tried the commands suggested above. The first one gives us many results (we couldn't get through all of it). If we try Image instead of OriginalFile, we got some images listed but not the images concerned. If we browse through the folder, we can see the images. I suppose it's not safe to delete them directly ? Is there a way to erase these images ?

Thank you again,
Daniel

[cblackburn]

Hi Daniel,

It's possible that the delete failed to remove the underlying image files for some reason. You could try to remove any of these remaining files using the cleanse command. It can be used in a dry-run mode to identify any files that need removing. If you can try:

Code: Select all

bin/omero admin cleanse --dry-run data_dir


as an admin with the full path to the OMERO data directory. If this does identify the files you tried to delete than running it without the dry-run option will attempt to delete those files.

Could you let us know, first, whether the command with the dry-run option does identify the files you expected to have been deleted? If it does then could you let us know whether a run without the dry-run option does succeed?

Cheers,

Colin

[dgarneau]

Hi,

We tried the cleanse command. The dry-run identified many files/folders but didn't identified the file we expected. We still tried the cleanse without the dry option but we still have these images. Now the cleanse --dry-run return nothing.

We really need to delete these files, is there any further thing we could try ?

Thanks,
Daniel

[jmoore]

Hi Daniel,

sorry for the troubles, but we'll need to figure out what's consuming the space to help you delete it. Could you show us the output of the CLI command `fs usage`:

Code: Select all

$ omero -q fs usage ExperimenterGroup:751 --report --human-readable
Total disk usage: 1.7 GB bytes in 4614 files
user | group | component    |     size | files
------+-------+--------------+----------+-------
3204 | 751   | Pixels       | 970.8 MB |   281
3204 | 751   | Annotation   | 294.7 KB |   284
3204 | 751   | Thumbnail    |   2.0 MB |   423
3204 | 751   | OriginalFile |   3.1 KB |     2
3251 | 751   | Thumbnail    |   1.7 MB |   906
3251 | 751   | Pixels       | 805.3 MB |  1812
3251 | 751   | Annotation   | 274.3 KB |   906
(7 rows)


Cheers,
~Josh.

[dgarneau]

Hi,
here are the results

Code: Select all

omero@omero3:~$ omero -q fs usage ExperimenterGroup:3 --report --human-readable Total disk usage: 175.0 GB bytes in 21591 files
user | group | component    |     size | files
------+-------+--------------+----------+-------
2    | 3     | FilesetEntry |   7.1 GB |    26
2    | 3     | Job          | 711.7 KB |    23
2    | 3     | Thumbnail    |  42.4 KB |    20
2    | 3     | Annotation   |  41.6 MB |     2
2    | 3     | OriginalFile |  17.3 KB |    28
803  | 3     | Job          |   3.9 KB |     1
803  | 3     | FilesetEntry | 167.9 GB | 21493
(7 rows)


omero@omero3:~$ omero -q fs usage Experimenter:803 --report --human-readable Total disk usage: 167.9 GB bytes in 21494 files
user | group | component    |     size | files
------+-------+--------------+----------+-------
803  | 3     | FilesetEntry | 167.9 GB | 21493
803  | 3     | Job          |   3.9 KB |     1
(2 rows)


Thanks for the help,
Daniel

[jmoore]

Hi Daniel,

do you know which of the two users you're looking to delete from? (bin/omero user list) Also, do you have a rough idea of when the import was performed?

Cheers,
~Josh.