Google Workspace - Google Drive

License: PRO+ (Professional Edition or higher)
Module Type: Alternate Scanner
Author: Diskover Data, Inc.

Overview / Use Cases

The Google Drive Alternate Scanner (scandir_google) gives Diskover visibility into your organization's Google Drive content by indexing files and folders through the Google Drive REST API (v3). It translates Google Drive's parent-child folder hierarchy into a POSIX-style path structure that Diskover can traverse, index, and search — turning cloud-resident data into searchable, analyzable metadata alongside your on-premises file system indexes.

The scanner handles personal drives ("My Drive"), shared drives (Team Drives), and direct folder-ID targeting via the gdrive-id:// URI scheme. Google Apps native files (Docs, Sheets, Slides, Forms, Drawings, Sites) are indexed with estimated sizes and virtual extensions (.gdoc, .gsheet, .gslides, etc.) so they appear alongside regular files in search results.

Key Use Cases

Cloud Storage Visibility (Storage Administrators & IT)
Files scattered across hundreds of user drives and dozens of shared drives are invisible at the filesystem level. The scanner indexes every accessible drive into Diskover, giving administrators a single search interface to locate files across the entire Google Workspace tenant without hunting through the Google Drive web UI.

Shared Drive Governance (Data Managers)
Shared drives (Team Drives) often accumulate years of content without clear ownership or lifecycle management. With include_shared_drives enabled, the scanner automatically discovers every shared drive and indexes its contents, making it easy to identify stale drives, oversized folders, and files that should be archived or deleted based on age, MIME type, or access patterns.

Cost & Capacity Planning (Storage Engineers)
The scanner queries Google Drive's storageQuota API to surface total and used storage, enabling unified capacity reporting across on-premises NAS and cloud-hosted Google Drive. Organizations can forecast storage growth, compare cost per TB across platforms, and make data-driven decisions about tiering or archival policies.

Understanding Google Drive & Google Cloud

Before configuring the scanner, it helps to understand a few Google-specific concepts that shape how the scanner is deployed and what it can see.

Drive Types

Authentication Methods

Recommendation: For production Diskover deployments running on servers, use a service account. Service accounts don't require interactive browser auth, tokens never expire, and they're easier to audit. OAuth 2.0 is suitable for development or single-user workstation use.

Google Apps File Types

Google Apps files (Docs, Sheets, Slides, etc.) are not traditional files — they live inside Google's infrastructure and have no native byte size. The scanner handles them with estimated sizes and synthetic extensions:

These can be excluded entirely by setting include_google_apps_files: false if you only care about traditional uploaded files (PDFs, images, Office docs, etc.).

Requirements

System Requirements

Python Dependencies

Google Cloud / Google Drive API Requirements

Installation

Step 1: Install the Scanner Package

Linux:

dnf install diskover-scanner-google

Windows:
The scanner files are included with the Diskover Windows installation — no separate install step is needed.

Install locations:

• Linux: /opt/diskover/scanners/scandir_google/

• Windows: C:\Program Files\Diskover\scanners\scandir_google\

Step 2: Install Python Dependencies

pip3 install -r /opt/diskover/scanners/scandir_google/etc/requirements.txt

Step 3: Verify Scanner Module Installation

Confirm the scanner module loads and dependencies resolve correctly:

cd /opt/diskover
python3 -c "import sys; sys.path.insert(0, 'scanners/scandir_google'); import scandir_google; print('Scanner module loaded OK, version:', scandir_google.__version__)"

Step 4: Configure Google Drive API Credentials

Choose one of the two authentication methods below.

Option A: Service Account (Recommended for Production)

Service accounts authenticate without browser interaction and are ideal for Diskover servers.

1. Go to Google Cloud Console and create (or select) a project.

2. Enable the Google Drive API: APIs & Services > Library > search "Google Drive API" > Enable.

3. Create a service account: APIs & Services > Credentials > Create Credentials > Service Account.

4. After creating the service account, open it and go to the Keys tab > Add Key > Create New Key > JSON. Download the key file.

5. Copy the key file to the scanner directory:

   cp ~/Downloads/<project>-<hash>.json /opt/diskover/scanners/scandir_google/service-account.json

6. Share the target drives/folders with the service account email (found in the service account details, e.g., diskover-scanner@my-project.iam.gserviceaccount.com). The service account can only see drives explicitly shared with it.

7. Set service_account_file: service-account.json in the Diskover Admin configuration (see Configuration).

Important: Service accounts cannot "see" drives by default — each drive, shared drive, or folder you want scanned must be explicitly shared with the service account's email address (Viewer permission is sufficient).

Option B: OAuth 2.0 (Interactive, Desktop Installs)

Use OAuth for ad-hoc scans from a workstation or for single-user deployments.

1. In Google Cloud Console, go to APIs & Services > Credentials.

2. Click Create Credentials > OAuth 2.0 Client ID > Desktop application.

3. Download the JSON file and save it as credentials.json in the scanner directory:

   cp ~/Downloads/client_secret_*.json /opt/diskover/scanners/scandir_google/credentials.json

4. On first run, a browser window opens asking you to grant read-only access to Google Drive. After approval, a token.json file is written for subsequent runs.

5. If running on a headless system, the scanner falls back to a manual URL-paste flow — visit the printed URL on another machine, approve, and paste the resulting code back into the terminal.

Step 5: Verify Google Drive Connectivity

Test that authentication works and the Drive API is reachable:

cd /opt/diskover
python3 -c "
import sys
sys.path.insert(0, 'scanners/scandir_google')
from google_scanner.drive_client.auth import GoogleDriveAuth
auth = GoogleDriveAuth(
    'scanners/scandir_google/credentials.json',
    'scanners/scandir_google/token.json',
    service_account_file='scanners/scandir_google/service-account.json'  # remove if using OAuth only
)
if auth.authenticate():
    svc = auth.get_service()
    about = svc.about().get(fields='user,storageQuota').execute()
    print('Authenticated as:', about['user']['emailAddress'])
    quota = about.get('storageQuota', {})
    print('Storage usage:', int(quota.get('usage', 0)) // (1024**3), 'GB')
else:
    print('Authentication failed')
"

Configuration

Configuration is managed through the Diskover Admin UI under Settings > Alternate Scanners > Google Drive.

Configuration Parameters

Advanced tuning: request_limit_per_minute and prefetch_workers rarely need adjustment. Increase prefetch_workers cautiously on large drives; decrease request_limit_per_minute if you share the Google Cloud project quota with other applications.

Configuration via Diskover Admin

1. Navigate to Settings > Alternate Scanners > Google Drive.

2. Set the credential file paths and choose your authentication method.

3. Enable or disable shared drives and Google Apps file inclusion.

4. Adjust timeout and rate-limit values if needed (defaults are appropriate for most deployments).

5. Save the configuration.

Configuration Examples

Example 1: Service Account on a Production Server (Recommended)

Diskover:
  Alternate Scanners:
    Google Drive:
      Default:
        credentials_file: ""
        token_file: ""
        service_account_file: service-account.json
        include_shared_drives: true
        include_google_apps_files: true
        request_timeout: 120
        request_limit_per_minute: 10000
        prefetch_workers: 10

Use this when Diskover runs on a headless server and you've shared all target drives/folders with the service account email.

Example 2: OAuth for a Workstation / Single-User Scan

Diskover:
  Alternate Scanners:
    Google Drive:
      Default:
        credentials_file: credentials.json
        token_file: token.json
        service_account_file: ""
        include_shared_drives: false
        include_google_apps_files: true
        request_timeout: 120
        request_limit_per_minute: 10000
        prefetch_workers: 10

Use this for ad-hoc scans from a workstation where a browser is available for first-time authorization.

Example 3: High-Volume Shared Drive Scan, Google Apps Excluded

Diskover:
  Alternate Scanners:
    Google Drive:
      Default:
        credentials_file: ""
        token_file: ""
        service_account_file: service-account.json
        include_shared_drives: true
        include_google_apps_files: false
        request_timeout: 180
        request_limit_per_minute: 8000
        prefetch_workers: 15

Use this for large tenants where you only care about uploaded files (PDFs, images, Office docs) and not Google Apps native documents. Reduced request_limit_per_minute leaves quota headroom for other Google API consumers.

Usage / Execution

The Google Drive scanner runs as a standard alternate scanner via the --altscanner flag with diskover.py.

Basic Usage

Scan the entire personal drive (root path is /gdrive):

# Linux
cd /opt/diskover
python3 diskover.py --altscanner scandir_google /gdrive

# Windows
cd "C:\Program Files\Diskover"
python diskover.py --altscanner scandir_google /gdrive

Scan a Specific Subfolder

python3 diskover.py --altscanner scandir_google /gdrive/Projects/2025

Scan by Google Drive Folder ID

When a folder name is ambiguous or hard to spell, use the gdrive-id:// URI to target it directly by its Drive folder ID:

python3 diskover.py --altscanner scandir_google gdrive-id://1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms

The folder ID is resolved to its name at initialization, and the index path becomes /gdrive/<folder_name>. The folder ID appears in the URL when you open a folder in the Google Drive web UI.

Include Shared Drives Automatically

With include_shared_drives: true in the configuration, scanning /gdrive automatically discovers and indexes every accessible shared drive as an additional top path (/gdrive/<drive_name>):

python3 diskover.py --altscanner scandir_google /gdrive

Path Format Reference

Advanced Usage Examples

Custom index name:

python3 diskover.py -i diskover-gdrive-2026 --altscanner scandir_google /gdrive

Verbose/debug logging:

python3 diskover.py --altscanner scandir_google --loglevel DEBUG /gdrive

Reduced thread count (lower API concurrency):

python3 diskover.py --altscanner scandir_google --threads 4 /gdrive

Integration with Index Tasks

The Google Drive scanner can be configured as part of a Diskover Index Task for scheduled/automated scans:

Performance Tips

• Start with defaults. The built-in rate limiter and prefetcher are tuned for typical workloads; adjust only if you see issues.

• Large tenants: Increase prefetch_workers to 15–20 for drives with thousands of subfolders. The background prefetcher fires files.list requests for child directories as soon as they're discovered, reducing wait time when the crawl pool reaches them.

• Shared quota environments: If other applications use the same Google Cloud project, reduce request_limit_per_minute to 6,000–8,000 to leave headroom.

• Thread tuning: The --threads flag controls Diskover's crawl thread count. For Google Drive, 4–8 threads is usually optimal; higher counts may trigger rate limiting without improving throughput.

• Excluding Google Apps files: If you only care about uploaded files, setting include_google_apps_files: false can significantly speed up scans of tenants with large numbers of Docs/Sheets.

Metadata Fields / Elasticsearch Mappings

In addition to standard file metadata (name, path, size, mtime, etc.), the scanner indexes four custom Google Drive-specific fields into Elasticsearch.

Field Mappings

Elasticsearch Mapping Definition

{
  "mappings": {
    "properties": {
      "gdrive_id": { "type": "keyword" },
      "gdrive_mimetype": { "type": "keyword" },
      "gdrive_webviewlink": { "type": "keyword" },
      "gdrive_drivename": { "type": "keyword" }
    }
  }
}

Example Indexed Document — Regular File

{
  "name": "quarterly_report.pdf",
  "path": "/gdrive/Finance/Reports/quarterly_report.pdf",
  "path_parent": "/gdrive/Finance/Reports",
  "extension": ".pdf",
  "size": 2097152,
  "size_du": 2097152,
  "mtime": "2026-01-15T10:30:00",
  "atime": "2026-01-15T10:30:00",
  "ctime": "2025-06-01T08:00:00",
  "type": "file",
  "gdrive_id": "1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms",
  "gdrive_mimetype": "application/pdf",
  "gdrive_webviewlink": "https://drive.google.com/file/d/1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms/view",
  "gdrive_drivename": "My Drive"
}

Example Indexed Document — Google Apps File

{
  "name": "Meeting Notes",
  "path": "/gdrive/Team/Meeting Notes",
  "path_parent": "/gdrive/Team",
  "extension": ".gdoc",
  "size": 51200,
  "size_du": 51200,
  "mtime": "2026-03-01T14:00:00",
  "atime": "2026-03-01T14:00:00",
  "ctime": "2025-10-15T09:00:00",
  "type": "file",
  "gdrive_id": "abc123def456ghi789",
  "gdrive_mimetype": "application/vnd.google-apps.document",
  "gdrive_webviewlink": "https://docs.google.com/document/d/abc123def456ghi789/edit",
  "gdrive_drivename": "My Drive"
}

Searching in Diskover

Use the Diskover Web UI search bar to query the custom Google Drive fields alongside standard file metadata.

Search Query Examples

Combined Queries

Troubleshooting

Debug Logging

Enable debug-level logging to trace authentication, API calls, and folder resolution:

python3 /opt/diskover/diskover.py --altscanner scandir_google --loglevel DEBUG /gdrive

Log File Locations

• Linux: /var/log/diskover/diskover.log

• Windows: Check Diskover service logs or the configured log location in Diskover Admin

Diagnostic Commands

Test authentication directly:

cd /opt/diskover
python3 -c "
import sys
sys.path.insert(0, 'scanners/scandir_google')
from google_scanner.drive_client.auth import GoogleDriveAuth
auth = GoogleDriveAuth(
    'scanners/scandir_google/credentials.json',
    'scanners/scandir_google/token.json'
)
print('Auth result:', auth.authenticate())
"

Check for rate limit warnings in a recent run:

python3 diskover.py --altscanner scandir_google --loglevel DEBUG /gdrive 2>&1 | grep -i "rate limit"

Test Google API reachability:

curl -s -o /dev/null -w "HTTP %{http_code}
" "https://www.googleapis.com/drive/v3/about"

2026-05-05 15:16:49,802 - diskover - INFO - Using alternate scanner scandir_google
2026-05-05 15:16:49,911 - google_scanner.drive_client.auth - INFO - Successfully authenticated using service account: /opt/diskover/scanners/scandir_google/service-account.json
2026-05-05 15:16:49,911 - diskover.scandir_google - INFO - Successfully authenticated with Google Drive API

Support

• Diskover Documentation

• Diskover Support Portal

• Google Drive API v3 Reference

• Google Cloud Console

• Google Drive API Quotas

• Service Account Overview

Last Updated: April 24, 2026
Diskover Data, Inc.