How to increase storage on a vRealize Log Insight cluster
Increasing the storage capacity on vRealize Log Insight is something that I get asked about fairly often, typically because a customer has a regulatory or business requirement to store log data for x number of weeks or months, and the customer still wants to be able to query the logs interactively. vRealize Log Insight uses its local storage like a rolling cache, which means once the available storage is close to full, vRealize Log Insight will purge the oldest log data in storage so it can store new log data.
This presents a use case which is distinct from the archive functionality within vRealize Log Insight, because the archived logs are no longer searchable. The archive function can be considered cold-storage of historical log data, and if archived log data needs to be queried at a future time, the recommendation is to deploy a separate vRealize Log Insight instance for that purpose.
Increasing the storage available to vRealize Log Insight is very easy even when the deployment is clustered, but lets go through how this process works.
If you take a look in the Administration > System Monitor view, you’ll see a number of performance statistics including a live storage bar at the bottom, which indicates the storage consumed on the appliance. If you have a clustered deployment, you’ll see a drop-down up the top for each node.
vRealize Log Insight doesn’t distribute data across clustered nodes for redundancy, which means in a clustered deployment you get an total storage capacity equalling the sum of the “total” storage size on each node, as shown in the live storage section at the bottom of of the System Monitor view.
My deployment comprises of three vRealize Log Insight nodes deployed with a “tiny” form factor, so each node provides 18.7 GB of storage for a total of just over 56 GB. I’ve only used ~700 MB of storage per node because this is a new deployment in a lab environment, but in a real deployment of any decent size this isn’t likely to last long.
Before you proceed, there are a few key things to know about adding storage to vRealize Log Insight nodes:
- You add storage by adding new disks to the VM, not by increasing the size of existing disks.
- You can add as many disks as you need, up to a toal of 4 TB of additional disks. This means you could add two 2 TB disks, four 1 TB disks, etc. Obviously, disks cannot be larger than 2 TB each, so you cannot add a single 4 TB disk.
- If you need more capacity than 4 TB per in the cluster, you will need to scale outwards by adding additional cluster nodes.
Before doing anything, I’ve used
df -h to list the mounted file systems on the host. /storage/core is the mount point used for log storage on vRealize Log Insight nodes.
1Filesystem Size Used Avail Use% Mounted on 2/dev/sda3 16G 1.9G 13G 13% / 3udev 2.0G 112K 2.0G 1% /dev 4tmpfs 2.0G 648K 2.0G 1% /dev/shm 5/dev/sda1 128M 39M 83M 32% /boot 6/dev/mapper/data-var 20G 654M 19G 4% /storage/var 7/dev/mapper/data-core 20G 733M 18G 4% /storage/core
To add storage, I shut down one of my nodes, add a 50 GB disk, and power it back on. During boot, the appliance will discover the new storage and add it to the default data volume.
df -h we can see that /storage/core has been extended across the new disk automatically.
1Filesystem Size Used Avail Use% Mounted on 2/dev/sda3 16G 1.9G 13G 13% / 3udev 2.0G 124K 2.0G 1% /dev 4tmpfs 2.0G 648K 2.0G 1% /dev/shm 5/dev/sda1 128M 39M 83M 32% /boot 6/dev/mapper/data-var 20G 650M 19G 4% /storage/var 7/dev/mapper/data-core 69G 716M 65G 2% /storage/core
In a clustered deployment, you would perform this operation one-by-one on each node in the cluster, ensuring that each node returns to a “connected” status in the Administration > Cluster view before moving on to the next node.
For more information, see the following docs.vmware.com pages:
comments powered by Disqus