Skip to content
Contact Support

GUFI

test

GUFI (Grand Unified File Index) is a file system metadata indexing tool designed for large-scale data centers to enable fast, secure, and comprehensive searches of files and directories. It works by creating a hierarchical index that preserves file access permissions, allowing users to efficiently find and characterize data across multiple, potentially disparate file systems. This results in significantly faster search times and lessens impact/load on parallel filesystems compared to traditional methods.

There are two commands that GUFI provides:

  • gufi_find: For finding files and subfolders in a directory
  • gufi_du: For obtaining the size of files and folders

Warning

This method uses a database that is updated on a weekly basis. It may not find or measure the size of files that were created or moved around Mahuika within the last week.

Note

The filesystems that gufi_find and gufi_du work on are:

  • \home
  • \projects
  • \nobackup

For gufi_find and gufi_du to work, you must give the path name beginning from project or nobackup. Local addresses starting with . will also not work. For example:

  • nobackup/nesi12345/a_folder and /nobackup/nesi12345/a_folder are acceptible, but
  • /nesi/nobackup/nesi12345/a_folder, nesi12345/a_folder, or ./nesi12345/a_folder will not work.

Prerequisite: Must Load gufi Module

To use gufi_find and gufi_du, you must load them by entering in the following command in Mahuika:

module load .gufi

Finding Files and Folders using GUFI

The usual method for searching for a file using the terminal is:

find path/to/folder/to/search -name <your_file>

In GUFI, you provide the same arguments to gufi_find as you do with find (but starting with either home, projects, or nobackup):

gufi_find path/to/folder/to/search -name <your_file>

Example

If you want to find .bashrc in your home directory:

gufi_find home/$USER -name .bashrc

If you want to find the largest file in your folder:

gufi_find home/$USER -type f -printf '%s %p\n' 2>/dev/null | sort -nr | head -n 1
gufi_find project/nesi12345 -type f -printf '%s %p\n' 2>/dev/null | sort -nr | head -n 1
gufi_find nobackup/nesi12345 -type f -printf '%s %p\n' 2>/dev/null | sort -nr | head -n 1

Obtaining the Size of Files and Folders using GUFI

The usual method for obtaining the size of a file or folder using the terminal is:

du -s path/to/file/or/folder

In GUFI, you provide the same arguments to gufi_du as you do with du (but starting with either home, projects, or nobackup):

gufi_du -s path/to/file/or/folder

For more options, see gufi_du --help

Example

If you want to find the size of your in your home directory:

gufi_du -s home/$USER

If you want to find the size of your .bashrc in your home directory:

gufi_du -s home/$USER/.bashrc

If you want to find the size of your project folder in projects or nobackup:

gufi_du -s project/nesi12345
gufi_du -s nobackup/nesi12345

If you want to obtain the number of files in a folder, such as nobackup/nesi12345/a_folder, you would do the following:

gufi_du --inodes -s nobackup/nesi12345/a_folder

Troubleshooting

If your file or folder exists but GUFI does not find it, it is likely that your folder has not been indexed by GUFI yet. You will need to wait until the end of the week for those files and folders to be indexed by GUFI.

  • In the meantime, you can either use find or du instead of gufi_find or gufi_du as alternatives for these functions.

Example

john.doe@login03:~$ gufi_find home/new_folder
Could not get realpath of "/search/home/new_folder": No such file or directory (2)

I get the message: Does "XYZ" have treesummary data?

If you get a message like this:

gufi_du --inodes -s nobackup/nesi99991
Error: Skipping directory "/search/nobackup/nesi99991": Permission denied (13)
0 nobackup/nesi99991
Warning: Did not get any results from gufi_query.
Does "nobackup/nesi99991" have treesummary data?

This means that gufi_find or gufi_du was not able to find any information about the path you gave it to search. This could be because:

  1. The folder you are search in doesn't exist,
  2. You don't have permissions to look at the files and folders that you were trying to search in, or
  3. Your files and folders were created before they were included in the GUFI database.

If 3 applies to you, you will need to wait until the end of the week for those files and folders to be indexed by GUFI.

I get the message: Error: Skipping directory "nobackup/XYZ": Permission denied (13)

If you see this error, this is because you do not have the correct permissions to view this directory.

  • If you want to use GUFI on this directory, you will need to get read permissions from the person who created this directory.

Example

john.doe@login03:/nesi/nobackup/nesi12345$ gufi_du nobackup/nesi12345/test.txt
Error: Skipping directory "nobackup/nesi12345/test.txt": Permission denied (13)

I can not use tab to autocomplete, and sometimes autocompleting using tab logs me out of my Mahuika login

This is a known problem. We are currently looking for a fix for this.