Xytech

Overview / Use Cases

The Xytech plugins integrate Diskover with the Xytech MediaPulse production management system, creating a powerful bridge between your file storage metadata and your production workflows. This integration consists of two complementary plugins that work together — or independently — to enrich your Diskover indices with valuable production data from MediaPulse.

Xytech Order Plugin — Scans completed Diskover indices for directories whose names match work order patterns (e.g., MO123456, 654321_ProjectName), queries the MediaPulse REST API for production metadata (phase status, customer info, scheduling, invoice data), and stores the results in a local cache. This cache is then consumed by the companion xytech attrs index plugin during the next Diskover scan to enrich directory documents in Elasticsearch with the retrieved metadata.

Xytech Asset Plugin — Scans completed Diskover indices for files that don't yet have a corresponding MediaPulse library asset, creates new assets via the MediaPulse SOAP API, and writes the resulting asset IDs and creation timestamps directly back to Elasticsearch.

Why Use the Xytech Plugins?

These plugins eliminate the need to manually cross-reference Diskover and MediaPulse, giving your teams a single pane of glass for both storage visibility and production context.

Common Use Cases

• Production Status Tracking — See which directories correspond to active, completed, or pending work orders without leaving Diskover. Teams can quickly identify where projects stand by searching for phase status, customer name, or due date directly in the Diskover UI.

• Automated Asset Registration — When new media files are ingested and indexed by Diskover, the Asset plugin automatically creates corresponding library assets in MediaPulse so production teams can immediately find and manage them through Xytech.

• Invoice Reconciliation — Link work order directories to their invoice records in MediaPulse, enabling finance teams to reconcile storage usage against billing data. Search for directories by invoice ID or invoice date directly within Diskover.

• Multi-Storage Batch Processing — Process multiple storage volumes in a single run to build a comprehensive metadata cache covering all work order directories across your infrastructure.

How the Plugins Work Together

Understanding the relationship between these plugins is important for getting the most out of the integration. Here's how the pieces fit together:

┌─────────────────────────────────────────────────────────────────────┐
│                        XYTECH ORDER PLUGIN                         │
│  (Post-Index — runs against a completed index)                     │
│                                                                    │
│  1. Scans Elasticsearch for directories matching work order names  │
│  2. Queries MediaPulse REST API for production metadata            │
│  3. Writes results to a local cache (keyed by directory path)      │
└──────────────────────────────┬──────────────────────────────────────┘
                               │
                               ▼
┌─────────────────────────────────────────────────────────────────────┐
│                     XYTECH ATTRS INDEX PLUGIN                      │
│  (Index Plugin — runs during the NEXT Diskover scan)               │
│                                                                    │
│  4. Reads the local cache during a Diskover scan                   │
│  5. Enriches directory documents in Elasticsearch with metadata    │
│     (phase, customer_name, account_manager, dates, invoice data)   │
└─────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────┐
│                       XYTECH ASSET PLUGIN                          │
│  (Post-Index — runs against a completed index)                     │
│                                                                    │
│  1. Queries Diskover for files missing xytech.asset_id             │
│  2. Extracts customer ID from each file's directory path           │
│  3. Creates library assets in MediaPulse via SOAP API              │
│  4. Writes xytech.asset_id and xytech.created_date back to ES     │
└─────────────────────────────────────────────────────────────────────┘

Key Point: The Order plugin and Asset plugin operate independently. You can use one without the other depending on your needs. However, the Order plugin requires the companion xytech attrs index plugin (an index-time plugin, documented separately) to actually write the cached metadata into Elasticsearch during the next scan.

Installation

Prerequisites

Python Dependencies

The Order plugin has no additional Python dependencies beyond what ships with Diskover.

The Asset plugin requires the suds-community package for SOAP API communication:

Linux:

python3 -m pip install suds-community

Windows:

python -m pip install suds-community

Plugin File Locations

Both plugins are installed in the Diskover post-index plugins directory:

Linux:

/opt/diskover/plugins_postindex/diskover_xytech_order/diskover_xytech_order.py
/opt/diskover/plugins_postindex/diskover_xytech_plugin/diskover_xytech_plugin.py

Windows:

C:\Program Files\Diskover\plugins_postindex\diskover_xytech_order\diskover_xytech_order.py
C:\Program Files\Diskover\plugins_postindex\diskover_xytech_plugin\diskover_xytech_plugin.py

The Asset plugin also includes a wsdl/ subdirectory containing the SOAP service definitions:

diskover_xytech_plugin/
  wsdl/
    XytechAPI.wsdl
    XytechAPI_0.xsd
    XytechAPI_1.xsd
    XytechAPI_2.xsd
    XytechAPI_3.xsd
    XytechAPI_4.xsd

Important: The WSDL file and all referenced XSD schema files must be co-located in the same directory, and the Diskover service user must have read access.

Verification

Run the following commands to verify your installation:

Linux:

# Verify Order plugin version
python3 /opt/diskover/plugins_postindex/diskover_xytech_order/diskover_xytech_order.py --version

# Verify Asset plugin version
python3 /opt/diskover/plugins_postindex/diskover_xytech_plugin/diskover_xytech_plugin.py --version

# Verify suds-community is installed (Asset plugin only)
python3 -c "from suds.client import Client; print('suds-community OK')"

# Verify Elasticsearch connectivity
python3 -c "from diskover_elasticsearch import es_connection_cached; print(es_connection_cached().info())"

# Test MediaPulse REST API connectivity (Order plugin)
curl -u username:password http://mp-server:8890/api/v1/database/VIZ_LIVE

Windows:

REM Verify Order plugin version
python "C:\Program Files\Diskover\plugins_postindex\diskover_xytech_order\diskover_xytech_order.py" --version

REM Verify Asset plugin version
python "C:\Program Files\Diskover\plugins_postindex\diskover_xytech_plugin\diskover_xytech_plugin.py" --version

REM Verify suds-community is installed (Asset plugin only)
python -c "from suds.client import Client; print('suds-community OK')"

Configuration

Both plugins are configured through the Diskover Admin UI. Navigate to Plugins > Post Index to find the configuration panels.

Xytech Order Plugin Configuration

Located under Plugins > Post Index > Xytech Order.

Configuration Parameters

Default Field Mappings

The fields_to_pull configuration defines which fields are extracted from the MediaPulse API and stored in the cache. Each mapping consists of an ESField (the Elasticsearch field name) and an XytechPath (a dot-notation path into the MediaPulse API response):

You can add custom field mappings to retrieve additional data from MediaPulse by specifying additional ESField / XytechPath pairs in the configuration.

Default Ignored Patterns

Regex Processing Flow

The plugin uses three layers of regex to discover and filter work order directories:

1. Elasticsearch Filter Regex — Used in the Elasticsearch query to find candidate directories. The default (MO)?([0-9]{6})(.*) matches directory names that optionally start with MO followed by a 6-digit number.

2. Ignored Regex — Any directory matching an ignored pattern is skipped entirely (e.g., UUIDs, hidden directories).

3. Parse Regex — An ordered list of patterns tried in sequence to extract the numeric work order ID. The first match wins. Directories that don't match any parse pattern are skipped.

Named Configurations

You can define multiple named configurations for different MediaPulse environments (e.g., production vs. development):

Plugins:
  Post Index:
    Xytech Order:
      Default:
        mediapulse_endpoint: http://mp-burbank-srv:8890
        mediapulse_db_name: VIZ_LIVE
        mediapulse_user_id: diskover-live
        mediapulse_password: prod_password
        workers: 5
      Dev:
        mediapulse_endpoint: http://mpdev-aws-west:8892
        mediapulse_db_name: VIZ_DEV
        mediapulse_user_id: diskover-dev
        mediapulse_password: dev_password
        workers: 2

Use the -c flag when running the plugin to select a named configuration (e.g., -c Dev).

Xytech Asset Plugin Configuration

Located under Plugins > Post Index > XytechAssetPlugin.

Configuration Parameters

Customer ID Extraction

The Asset plugin derives a customer identifier from each file's parent directory path. It takes the last component of the path and applies the customer_id_regex pattern.

Default regex: ([^_/]+)_ — Captures everything before the first underscore.

Files whose parent directory does not match the regex are silently skipped.

WSDL Setup

The Asset plugin communicates with MediaPulse through a SOAP interface defined by WSDL and XSD schema files. The mp_wsdl_path can point to either a local file or a remote URL:

• Local file: /opt/diskover/plugins_postindex/diskover_xytech_plugin/wsdl/XytechAPI.wsdl

• Remote URL: https://mediapulse.example.com/services/XytechAPI?wsdl (uses authenticated HTTP transport with the configured credentials)

Execution / Usage Guide

Running the Xytech Order Plugin

Manual Execution

Linux:

# Process a specific index
python3 /opt/diskover/plugins_postindex/diskover_xytech_order/diskover_xytech_order.py diskover-myindex-1234567890

# Auto-find the latest index for a top path
python3 /opt/diskover/plugins_postindex/diskover_xytech_order/diskover_xytech_order.py -l /mnt/storage1

# Auto-find latest indices for multiple top paths
python3 /opt/diskover/plugins_postindex/diskover_xytech_order/diskover_xytech_order.py -l /mnt/storage1 -l /mnt/storage2 -l /mnt/storage3

# Use a named configuration with verbose output
python3 /opt/diskover/plugins_postindex/diskover_xytech_order/diskover_xytech_order.py -c Dev -v -l /mnt/storage1

Windows:

REM Process a specific index
python "C:\Program Files\Diskover\plugins_postindex\diskover_xytech_order\diskover_xytech_order.py" diskover-myindex-1234567890

REM Auto-find the latest index for a top path
python "C:\Program Files\Diskover\plugins_postindex\diskover_xytech_order\diskover_xytech_order.py" -l /mnt/storage1

REM Use a named configuration with verbose output
python "C:\Program Files\Diskover\plugins_postindex\diskover_xytech_order\diskover_xytech_order.py" -c Dev -v -l /mnt/storage1

Command-Line Options (Order Plugin)

Note: You must provide either positional index name arguments or one or more -l TOPPATH options. If neither is specified, the plugin exits.

Running the Xytech Asset Plugin

Manual Execution

Linux:

# Basic execution against a specific index
python3 /opt/diskover/plugins_postindex/diskover_xytech_plugin/diskover_xytech_plugin.py diskover-myindex

# Dry run to validate configuration without making changes
python3 /opt/diskover/plugins_postindex/diskover_xytech_plugin/diskover_xytech_plugin.py --dry-run -v diskover-myindex

# Use a named configuration with verbose output
python3 /opt/diskover/plugins_postindex/diskover_xytech_plugin/diskover_xytech_plugin.py -c production -v diskover-myindex

Windows:

REM Basic execution against a specific index
python "C:\Program Files\Diskover\plugins_postindex\diskover_xytech_plugin\diskover_xytech_plugin.py" diskover-myindex

REM Dry run to validate configuration
python "C:\Program Files\Diskover\plugins_postindex\diskover_xytech_plugin\diskover_xytech_plugin.py" --dry-run -v diskover-myindex

Command-Line Options (Asset Plugin)

Tip: Use --dry-run mode before running against a large production index to verify connectivity, credential validity, and the number of files that will be processed.

[root@dss-250 diskover_xytech_order]# python3 diskover_xytech_order.py diskover-xytech
2026-04-04 22:50:18,237 - diskover.plugin.xytech_order - INFO - Starting diskover xytech_order ...
2026-04-04 22:50:18,237 - diskover.plugin.xytech_order - INFO - --- Connecting to Diskover Elasticsearch service...
2026-04-04 22:50:18,238 - diskover.plugin.xytech_order - INFO - Initializing cache at /tmp/__xytech_order_cache__/
2026-04-04 22:50:18,238 - diskover.cache - INFO - Using cache DB /tmp/__xytech_order_cache__/cache_database.db (16.0 KB)
2026-04-04 22:50:18,238 - diskover.plugin.xytech_order - INFO - Cache initialized successfully at /tmp/__xytech_order_cache__/
2026-04-04 22:50:18,239 - diskover.plugin.xytech_order - ERROR - Cache test failed: 'Cache' object has no attribute 'delete_value'
2026-04-04 22:50:18,239 - diskover.plugin.xytech_order - INFO - --- Starting script...
. . .
2026-04-04 22:50:18,252 - diskover.plugin.xytech_order - INFO - Processing: (1/3) 456789:/tmp/test_data/456789_Delta_Project
2026-04-04 22:50:18,252 - diskover.plugin.xytech_order - INFO - Processing: (2/3) 567890:/tmp/test_data/567890_Echo_Campaign
2026-04-04 22:50:18,253 - diskover.plugin.xytech_order - INFO - Processing: (3/3) 678901:/tmp/test_data/678901_Foxtrot_Show
2026-04-04 22:50:18,254 - diskover.plugin.xytech_order - INFO - Getting invoice ID for Invoice no INV-2026-679 and work order 567890....
2026-04-04 22:50:18,256 - diskover.plugin.xytech_order - INFO - -> Retrieved invoice ID: DOC-21902
2026-04-04 22:50:18,256 - diskover.plugin.xytech_order - INFO - Getting invoice ID for Invoice no INV-2026-980 and work order 678901....
2026-04-04 22:50:18,257 - diskover.plugin.xytech_order - INFO - -> Retrieved invoice ID: DOC-14758
2026-04-04 22:50:18,257 - diskover.plugin.xytech_order - INFO - Cache statistics: Cache stats not available
2026-04-04 22:50:18,257 - diskover.cache - INFO - CACHE HITS: 1, MISSES: 0, HIT RATIO: 100.0% (/tmp/__xytech_order_cache__/)
2026-04-04 22:50:18,257 - diskover.cache - INFO - Closing cache DB /tmp/__xytech_order_cache__/cache_database.db...
2026-04-04 22:50:18,260 - diskover.cache - INFO - Cache DB /tmp/__xytech_order_cache__/cache_database.db closed
2026-04-04 22:50:18,260 - diskover.plugin.xytech_order - INFO - Cache database closed
2026-04-04 22:50:18,260 - diskover.plugin.xytech_order - INFO - === FINAL PROCESSING SUMMARY ===
2026-04-04 22:50:18,260 - diskover.plugin.xytech_order - INFO - *** Elapsed time 0.02 seconds ***
2026-04-04 22:50:18,260 - diskover.plugin.xytech_order - INFO - *** Total media orders found: 3 ***
2026-04-04 22:50:18,260 - diskover.plugin.xytech_order - INFO - *** Successfully processed: 3 ***
2026-04-04 22:50:18,260 - diskover.plugin.xytech_order - INFO - *** Errors: 0 ***
2026-04-04 22:50:18,260 - diskover.plugin.xytech_order - INFO - *** Success rate: 100.0% ***
2026-04-04 22:50:18,260 - diskover.plugin.xytech_order - INFO - *** get_work_order function cache CacheInfo(hits=0, misses=3, maxsize=None, currsize=3) ***
2026-04-04 22:50:18,260 - diskover.plugin.xytech_order - INFO - *** get_invoice function cache CacheInfo(hits=0, misses=2, maxsize=None, currsize=2) ***
2026-04-04 22:50:18,260 - diskover.plugin.xytech_order - INFO - Processing completed successfully with no errors.

Automated Execution

Both Xytech plugins can be automated within Diskover using Custom Tasks or Post-Crawl Commands of an Index Task. Do not use crontab or local cron/Task Scheduler configurations.

Custom Task

Schedule either plugin as a Custom Task through the Diskover Admin UI. This allows you to define recurring execution schedules and monitor task status.

Post-Crawl Command (Order Plugin)

Configure the Order plugin to run automatically after an index completes by adding it as a Post-Crawl Command to an Index Task:

Linux Example:

Windows Example:

Post-Crawl Command (Asset Plugin)

Linux Example:

Windows Example:

Available Index Task Tokens:

• {indexname} — The name of the index that was just created

Important:

• The Post-Crawl Command field should contain ONLY the executable (e.g., python3, python)

• All script paths, flags, and arguments go in the Post-Crawl Command Args field

• Never reference crontab or local cron/Task Scheduler configurations for automated execution

Reviewing the Output

Xytech Order Plugin Output

During execution, the Order plugin logs progress including:

• The number of directories matching the work order regex

• Per-directory processing status showing the work order ID and path

• Invoice lookup results (when configured)

• A final processing summary with elapsed time, success count, error count, and API cache hit statistics

Example summary output:

=== FINAL PROCESSING SUMMARY ===
*** Elapsed time 45.23 seconds ***
*** Total media orders found: 342 ***
*** Successfully processed: 338 ***
*** Errors: 4 ***
*** Success rate: 98.8% ***
*** get_work_order function cache CacheInfo(hits=52, misses=290, maxsize=None, currsize=290) ***
*** get_invoice function cache CacheInfo(hits=15, misses=275, maxsize=None, currsize=275) ***

The LRU cache statistics indicate how many duplicate work order IDs were found — a high hit count means the plugin efficiently avoided redundant API calls.

The output is written to the local cache directory (configured via cache_dir). This cache is consumed by the xytech attrs index plugin during the next Diskover scan.

Xytech Asset Plugin Output

The Asset plugin logs:

• The number of files found without Xytech asset IDs

• Per-batch processing status

• Individual asset creation results

• Elasticsearch bulk update confirmations

After execution, the xytech.asset_id and xytech.created_date fields are written directly to the Elasticsearch documents for each processed file.

Searching in Diskover

Once the plugins have run and metadata has been enriched (either through the Order → attrs pipeline or directly via the Asset plugin), you can search for the new fields in the Diskover web UI.

Fields Added by the Xytech Attrs Index Plugin (via Order Plugin Cache)

These fields appear on directory documents after the Order plugin builds the cache and the xytech attrs index plugin enriches the index on the next scan:

Fields Added by the Xytech Asset Plugin

These fields appear on file documents after the Asset plugin runs:

Example Search Queries

# Find all directories with active work orders
xytech.phase:"In Progress"

# Find all completed work orders for a specific customer
xytech.phase:Completed AND xytech.customer_name:ACME

# Find work orders due this quarter
xytech.due_date:[2026-04-01 TO 2026-06-30]

# Find directories with invoices
xytech.invoice_id:* AND NOT xytech.invoice_id:"n/a"

# Find files that have been registered as MediaPulse assets
xytech.asset_id:*

# Find files that have NOT been registered as assets yet
type:file AND NOT xytech.asset_id:*

# Find assets by specific asset ID
xytech.asset_id:12345

# Combine Xytech data with standard Diskover searches
xytech.asset_id:* AND extension:mov AND size:>1073741824

# Find assets created in a specific date range
xytech.created_date:[2026-02-01 TO 2026-02-28]

# Find work orders managed by a specific account manager
xytech.account_manager:"Jane Doe"

Troubleshooting

MediaPulse REST API Connection Failures (Order Plugin)

Symptom: Connection error while retrieving order ID or MediaPulse connectivity test failed in logs.

Resolution:

• Verify the mediapulse_endpoint URL is correct and includes protocol and port (e.g., http://mp-server:8890)

• Confirm mediapulse_db_name matches the target MediaPulse database

• Check firewall rules allow traffic from the Diskover host to the MediaPulse port

• Test connectivity directly: curl -u username:password http://mp-server:8890/api/v1/database/VIZ_LIVE

Authentication Errors (Order Plugin)

Symptom: HTTP Error 401 or HTTP Error 403 in logs.

Resolution:

• Verify mediapulse_user_id and mediapulse_password are correct

• Confirm the user has read access to MoMediaOrder and JmInvoice API resources

• Check if the user account is locked or expired in MediaPulse

No Work Orders Found (Order Plugin)

Symptom: No media orders found to process. Exiting. in logs.

Resolution:

• Verify elasticsearch_filter_regex matches your directory naming convention

• Check that elasticsearch_parse_regex patterns can extract IDs from matched names

• Review ignored_regex to ensure desired directories are not being filtered out

• Run with -v for verbose logging to see per-directory filtering details

WSDL File Not Found (Asset Plugin)

Symptom: Failed to initialize SOAP client error at startup.

Resolution:

• Verify mp_wsdl_path points to the correct location of XytechAPI.wsdl

• Ensure all six WSDL/XSD files are present in the same directory

• Confirm the Diskover service user has read access to these files

No Assets Found to Process (Asset Plugin)

Symptom: No assets found that need Xytech IDs even though files exist.

Resolution:

• Verify the Diskover API is reachable at the configured diskover_endpoint

• Check if files already have xytech.asset_id from a previous run

• Verify customer_id_regex matches your directory naming convention

Enabling Debug Logging

Both plugins support verbose debug logging via the -v flag:

Linux:

python3 /opt/diskover/plugins_postindex/diskover_xytech_order/diskover_xytech_order.py -v diskover-myindex

python3 /opt/diskover/plugins_postindex/diskover_xytech_plugin/diskover_xytech_plugin.py -v diskover-myindex

Windows:

python "C:\Program Files\Diskover\plugins_postindex\diskover_xytech_order\diskover_xytech_order.py" -v diskover-myindex

python "C:\Program Files\Diskover\plugins_postindex\diskover_xytech_plugin\diskover_xytech_plugin.py" -v diskover-myindex

The Order plugin also supports file-based logging by setting log_to_file: true and configuring log_path in the plugin configuration.

Support

• Diskover Documentation

• Diskover Support Portal

Last Updated: April 2026
Diskover Data, Inc.