Importing the OVA
Diskover provides an OVA build for WMware. This OVA can easily be imported into any WMware virtualization product and be used to run Diskover! Below are some light instructions on how to import the OVA properly and get it booted up.
(WMware Workstation Import) - To import the OVA you will choose File -> Open - Give the VM a unique name - ex: 'diskover-2.6.0' -- Once you've given the VM a name, choose 'Import' (WMware Workstation Adjust VM Settings) By default the OVA sets the CPU=4 and RAM=16GB - Adjust these settings to something applicable for your host machine -- Recommended Specs for OVA usage for POC : CPU=12 and RAM=64GB - Adjust the disk space allocation to at least 250GB -- Once you've adjusted the settings choose 'Power on this virtual machine' (OVA Login) - You will see two user accounts 'Vagrant' and 'Diskover' - Choose to login as the 'Diskover' user -- Password 'darkdata'
Bootstrap Wait Period
The Diskover OVA changed significantly in the 2.6.0 release. Because of these changes, the system needs a backend bootstrap while the OVA boots. Wait 15–20 minutes after the OVA has been imported and powered on to the login screen before interacting with the system. Here is what is happening behind the scenes and some tests you can run to ensure that the system is fully ready for usage:
Elasticsearch + Kibana SSL Configuration - This cannot be done during the OVA build process as there has not been an IP Address assigned to the host. The SSL certs for Elasticsearch need to be bound to the IP Address so they can be trusted by an outside source and referenced by IP. If we were to bootstrap the Elasticsearch SSL during the build process, the certs would only be bound to
localhostand we could not use a remote Diskover Task Worker to connect to this system.Elastic Agent + Fleet Configurations - This requires the above to be completed before it can be built. This manages the log search / aggregation that was introduced in Diskover 2.6.0!
Here are a couple of helpful commands you can run to validate that this has been completed:
(Activies)
- Click on 'Activities' in the top left
-- Open the terminal app
-- Run 'sudo -i' to escalate to the 'root' user
(Check Elasticsearch Cluster Health)
- get the 'elastic' user password
cat /root/.config/elastic.txt (needed for the $ES_PASSWORD variable below)
- curl for ES health
curl https://$OVA_IP_ADDRESS:9200/_cluster/health?pretty -u "elastic:$ES_PASSWORD" --cacert /etc/Elasticsearch/certs/http_ca.crt
-- This should return a happy response similar to the following, both green and yellow statuses are ok here:
{
"cluster_name" : "diskover",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 35,
"active_shards" : 35,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"unassigned_primary_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}
(Check Elastic-Agent Status)
elastic-agent status
┌─ fleet
│ └─ status: (HEALTHY) Connected
└─ elastic-agent
└─ status: (HEALTHY) Running
(Check PostgreSQL Status)
- Check the database service is running
systemctl status postgresql-18
* Should show 'active (running)'
- Confirm it's accepting connections
/usr/pgsql-18/bin/pg_isready -h localhost -p 5432
* Should return 'localhost:5432 - accepting connections'
If all of the preceding commands are returning healthy responses, you’re good to go to the next steps here!
Diskover Front-End Configurations
Now that we have the OVA imported into our hypervisor we have a bit of configuration to do here to get the system up and running and ready to scan.
(Diskover Configuration Wizard)
2.6.0 has an easy configuration wizard and most of the details are pre-filled for us here!
- In the browser window go to : http://$OVA_IP_ADDRESS:8000
-- Default login is username: admin & password: darkdata
(Diskover Configuration Wizard - Elasticsearch)
- In the initial section here you will see 'hosts-0' and within the box you will see 'localhost', replace this with the IP of the OVA
-- Choose 'Test' in the bottom right
* This should show your ES cluster health with a status of 'green' at the top of the window.
- Choose 'Save & Continue'
- Choose 'No' when asked if you want to "Copy connection settings from Diskover Web"
-- Choose 'Test' in the bottom right
* This should show your ES cluster health with a status of 'green' at the top of the window.
- Choose 'Save & Continue'
(Diskover Configuration Wizard - License)
- You will need to request a license here from the Diskover Data team
-- You can grab the "System Hardware ID is xxxxxxxxxxxxxxx" value and send it to licenses@diskoverdata.com separately!
- Once available copy the contents of the two files into their respective fields in the UI and 'save'
(Diskover Configuration Wizard - Time Settings)
- Select your proper timezone
- Choose 'Save & Continue'
- Check the box for 'Show Times in Local Timezone'
- Choose 'Save & Continue'
(Diskover Configuration Wizard - RabbitMQ)
- Change 'localhost' at the bottom to the $OVA_IP_ADDRESS
-- Choose 'Test' in the bottom right
* This should show 'Connection Successful' at the top of the screen
(Diskover Configuration Wizard - API)
- Choose 'Save & Continue'
Diskover Back-End Configurations
Now our Diskover system is fully configured and we just need to start the Task Worker service!
(Start the diskoverd Service) systemctl start diskoverd
Scanning Sample Storage
This section will be a real quick test scan of the local OVA filesystem. To properly scan your own storage you will need to set up and configure SMB / NFS shares to be mounted to the OVA. Once these shares are mounted you can follow a similar process as outlined below to scan those storage repositories!
(Login to Diskover)
Go to - http://$OVA_IP_ADDRESS:8000 and login as the 'admin' user
Username: admin
Password: darkdata
* Once you input this password you will be prompted to reset the password and log back in with your own password
(Create a Task)
On this next page choose 'Schedule Index Task'
On this next page choose 'New Index Task' - button near the top left of the screen
* We will only change 5 fields in this default template to scan our local storage
Name - set to some descriptive name (ex: Diskover Build Directory)
Description - set some more detailed description of the task (ex: Scan of the local /opt/diskover directory)
Crawl Directory(s) - this will be the path available to the OVA that we want to scan (ex: /opt/diskover)
Scanner - Set this to 'DirCache'
Custom Index Name - we want to differentiate each task's index name so that they're never the same (ex: diskover-opt-%Y%m%d%H%M)
Custom Schedule - let's just set this to 1AM Daily - 0 1 * * *
Disabled - we will want to just uncheck this box!
* Once we've set all these properties we can choose 'Create Task' at the bottom of the page
(Review the Log Files)
* Before we start our scan task, let's tail the log files so we can watch progress of our scan!
tail -F /var/log/diskover/*
(Start a Scan)
* From the task list page - http://$OVA_IP_ADDRESS:8000/tasks/index.php
Choose your newly created task and select the arrow next to the 'Info' button and choose 'Run Now' and click 'Ok'
* Go back to your terminal and watch the scan logs
Reviewing the Sample Index
Now that we have scanned the test directory let’s take a look at that index that got created from this storage location!
(Go to Indices page)
In the top right corner choose the settings cog dropdown and select 'Indices'
* Or go directly to http://$OVA_IP_ADDRESS:8000/selectindices.php
In the top left side of the screen you will see 'Always use latest indices (auto select)' ensure this checkbox is selected
(Go to File Search page)
In the top left corner choose the folder icon
* You can now review your scan of your test folder and see / search across the contents within
Helpful Notes
Within the OVA itself if you’re using it as a self-contained image and not accessing the OVA over a bridged network outside of the OVA itself it can be helpful to bookmark a couple of links inside of the OVA so it’s easy to get to these things:
Diskover - http://$OVA_IP_ADDRESS:8000
Diskover Admin - http://$OVA_IP_ADDRESS:8000/diskover_admin/config/
Kibana - http://$OVA_IP_ADDRESS:5601
RabbitMQ - http://$OVA_IP_ADDRESS:15672
Comments
0 comments
Please sign in to leave a comment.