These terms may have different meanings in other contexts, but the definitions below apply specifically to the Diskover user environment.
➲ Data Curation
Data curation refers to managing data from various sources as a valuable asset. It involves having a clear data strategy and reliable methods to access, integrate, cleanse, govern, store, and prepare data for analytics. Effective curation ensures your data stays useful over time and remains available for reuse and long-term preservation.
➲ Directory/Folder
Although “directory” and “folder” can differ slightly in other contexts, in this guide they mean the same thing: a container used to organize files and other directories.
Learn how to use the directories in the user interface →
➲ Elasticsearch
Elasticsearch powers Diskover’s search and analytics engine. It stores indexed metadata and enables fast queries, reports, and visualizations across large storage environments.
Learn more about Diskover’s architecture →
Hardlinks
A hardlink is an additional filename that refers to the same underlying file data as another filename on the same filesystem. Neither filename is the “original”—both refer to the same inode and content.
Renaming, moving, or deleting one hardlink does not break the others, provided the file remains on the same filesystem. The file data is removed only after its last hardlink is deleted and no process still has the file open.
Unlike a symbolic link (also called a soft link), a hardlink does not store the path to another file. Hardlinks cannot cross filesystem boundaries, and hardlinks to directories are generally not permitted.
✏️ Hardlinks are commonly used in the media and entertainment industry to reference the same digital asset from multiple shot folders without duplicating its data.
Feature |
Symbolic Link (soft link) |
Hardlink |
|---|---|---|
Can cross filesystem boundaries |
Yes |
No |
Can point to a directory |
Yes |
Generally no |
Inode |
Has its own inode |
Uses the same inode as the other handlinks |
Permssions |
Access is normally governed by the target’s permissions |
Permissions are shared because all hardlinks refer to the same inode |
What it stores |
The path to another file or directory |
A directory entry referring to the same inode and file data |
If another filename is moved or renamed |
The link may break because its stored path is no longer valid |
The remaining hardlinks continue to work |
If another filename is deleted |
The link becomes invalid if it pinted to that name |
The remaining hardlinks continue to work |
Disk space |
Uses a small amount of space to store the link |
Does not duplicate the file data; only the additional directory entry uses space |
➲ Hash Values vs Inodes
Inodes and hash values both identify files, but in different ways and for different purposes, such as search and deduplication.
🔆 When to use what:
Use inode when tracking a file on the same system (even if renamed or moved).
Use hashes when comparing file content across systems or locations.
DESCRIPTION |
INODE |
HASH VALUE |
|---|---|---|
What it represents |
An identifier assigned to a file or directory within a filesystem. |
A content-derived fingerprint calculated from the file’s data. |
How it works |
Each file or directory is assigned an inode number by the filesystem. |
A hash function generates a fixed-length value from the file’s content. |
Changes when… |
The inode normally stays the same when a file is renamed or moved within the same filesystem. Moving it to another filesystem creates a new inode. |
File content changes ➝ Hash changes. |
Used for |
Tracking files within the same filesystem (including hardlinks). |
Verifying file integrity and detecting duplicates across different storage locations. |
Diskover search field |
|
|
Key limitation |
Inode numbers are unique only within a single filesystem—same file copied to another storage system gets a new inode. |
A hash identifies content, not a particular file. Files with identical content have the same hash even when their names, dates, or locations differ. Hash collisions are also theoretically possible. |
➲ Index/Indexes/Indices
An index is a searchable inventory of all metadata (attributes) about files within a volume. Instead of searching through the operating system directly, Diskover searches through the index for faster results.
Both indexes and indices are correct plural forms; indices is more common in technical contexts.
You can have multiple indices (snapshots/inventories) of the same storage volume from different points in time. Indices contain core metadata such as directory name, file name, size, extension, creation date, modification date, owner, and more. Diskover can also add enriched metadata, which provides business context.
➲ Metadata/Attributes
Metadata is information about a file—the details that describe what it is, where it is located, who owns it, and more. In Diskover, metadata falls into two main categories:
Core Metadata (always collected):
These are the basic details your file system already knows, such as:
File name
Size
Type/extension
Timestamps (last accessed, last modified, and last changed; creation time where supported)
Path/where it’s located
Owner and permissions
Extra Metadata (added via plugins):
This is additional information that gives files business context, such as:
Customer or asset details
Media-specific metadata (codec, duration, resolution, etc.)
Metadata extracted from scientific file formats such as BAM and SAM
Diskover stores and organizes metadata so you can search, filter, analyze, and automate using factual information about your files — instead of manually tracking everything yourself.
➲ Path
A path is the full location of a directory or file, for example:
/mnt/lucidlink/projects/Pistachio/WonderfulPistachios_GangnamStyle.mov
➲ Recursive and Non-Recursive
Non-Recursive: A non-recursive action applies only to the items directly inside the selected path. It does not include any sub-folders or their contents.
Recursive: A recursive action applies to everything inside the selected path, including all sub-folders and their files at every level.
➲ Size
Definition by field name:
size: The default reported file size. This is the standard “logical size” shown in most Diskover views.
size_du: The allocated size on disk—how much storage the file actually takes up based on filesystem block usage. Useful for capacity planning.
file_size: Another logical size field that behaves like
size. If you see bothsizeandfile_sizein your environment, check with your System Administrator which one applies to your setup.file_size_du: Allocated size for file items only. Similar to
size_dubut excludes directories.pscale.size_logical: Logical file size reported specifically from Dell PowerScale.
pscale.size_physical_data: The actual physical storage consumed on PowerScale, accounting for protection overhead and layout. Most accurate for true footprint on PowerScale systems.
pscale.size_protection: Amount of space used for protection overhead (mirroring, erasure coding, etc.) on PowerScale.
size_du_norecurs: Allocated size without including subfolders. Best used when you want the true size of a single folder level.
size_logical: Logical file size (same meaning as
size), but pulled from a different metadata field.size_norecurs: Logical size of a directory without counting the contents of subfolders.
size_physical_data: The actual bytes stored on disk, reported by supported scanners. Not available in all environments—contact your System Administrator if you don't see this field.
size_protection: The storage space used for data protection overhead (such as mirroring or erasure coding), reported by supported scanners. Not available in all environments—contact your System Administrator if you don't see this field.
🔆 By category:
Standard size fields:
size,size_du,size_logical,file_size,file_size_duNon-recursive fields:
size_norecurs,size_du_norecursDell PowerScale fields:
pscale.size_logical,pscale.size_physical_data,pscale.size_protectionVendor-specific / extended fields:
size_physical_data,size_protection
➲ Stemming
Stemming is applied to .text fields in Diskover searches. It allows Diskover to match different forms of the same word automatically (e.g., run, running, runner).
⚠️ Stemming only applies to .text fields (such as name.text). Fields like name and path do not use stemming.
HOW IT WORKS |
EXAMPLES |
WHAT THIS MEANS FOR USERS |
|---|---|---|
|
Stemming matches words with the same root, so you don’t need to search for every variation. Instead of searching for the exact word, Diskover looks for related words with the same root. |
|
|
➲ Timestamps
atime → last accessed: Records when the file was last accessed. Depending on the filesystem and its configuration,
atimemay not update after every read.ctime → last changed: Records when the file’s content or metadata changes. Metadata changes can include changes to permissions, ownership, or hardlinks. Despite its name,
ctimeis not the file’s creation time.mtime → last modified: Records when the file’s contents were last changed. Changing metadata such as permissions or ownership does not normally update
mtime, but it does updatectime.
➲ Volume
A volume is any storage location indexed by Diskover—for example, a Windows share or drive, a Linux mount/NFS export, cloud storage like an S3 bucket, Dell PowerScale storage, a laptop, or even a USB drive.
Your organization may refer to volumes using other names, such as storage volume, mount, mount point, top-level path, or top-level storage.
Comments
0 comments
Please sign in to leave a comment.