vRealize Log Insight Internal Certificate issue

Issue

A KB91441 was recently published that affects vRLI internal certificate that will expire on April 30th. The expiration on the certificate will leading to a comunication failures in internode comunication in a vRLI cluster. However, this problem affects both cluster and single instance installation of a vRLI, so I invite you to read carefully the KB and act as soon as possible.

Before to act on the customer's production cluster environment I prefer to do some tests on the safe VMware HOL environment, using this LAB "Getting Started with Aria Operations for Logs (HOL-2201-03-CMP)".

Let's see below the tasks, following the KB "Updating the vRealize Log Insight Internal Certificate (91441)" step by step.

Solution

As the KB says:

This is a known issue affecting vRealize Log Insight 8.x.
The official resolution will be in the next vRealize Log Insight due out before April 30th 2023.

In my case the vRLI is a single instance version 8.4.

Below the verification steps before the fix:

1. Let's check the external certificates, also because the problem may not arise with the expiration of the external certificate, but it is actually the internal certificate that expires:
   
   Open a web browser and open the vRLI URL and verify the certificate
   
   
   
   
2. Get access to the vRealize Log Insight with user admin (Default built-in) > Administration > SSL.
   
   Click on “VIEW DETAILS…” of Existing Certificate
   
   
   
   As you can see here, the external certificates don't seems to expire imminently
   
   
3. Connect via SSH to the vRealize Log Insight appliance with root user and run the following command:
   
   

   # openssl x509 -in /storage/core/loginsight/cidata/cassandra/config/cacert.pem -enddate -noout

   
   If the result is as indicated in the figure above, it must be updated, even if in points 1. and 2. the certificate is not expired. If the result is different and expires beyond April 30, 2023, no certificate reconfiguration activities are required.
   
   

Below the workaround:

1. Take a cold snapshot of the vRealize Log Insight VM
   
   
   
2. Generating the new self-signed certificate:
   

   openssl req -newkey rsa:2048 -keyout domain.key -x509 -days 3650 -out domain.crt -nodes 

   When prompted by openssl, provide the required values for your company.
   
   Then run the following command to concatenate the key and cert into a pem file

   cat domain.key domain.crt > /tmp/cert.pem

   
3. Download the cert.pem file and upload it to vRealize Log Insight:
   Navigate to Configuration > SSL, click Choose File, browse to the cert.pem file previusly downloaded and click Open.
   Click Save. This will automatically distribute the new cert across all nodes in the vRealize Log Insight cluster. Wait for the SSL certificate to be updated.
   
   
4. Run the "update_default_cert.sh" script downloaded from VMware KB:

   # ./update_default_cert.sh --all

   Once completed, stop the loginsight service on the node by running the following command:

   # systemctl stop loginsight

   Start the loginsight service by running the following command:

   # systemctl start loginsight

   Execute the script with the verify option:

   # ./update_default_cert.sh --verify

   
5. Run the following command to validate that the new certificate is in place:

   # openssl x509 -in /storage/core/loginsight/cidata/cassandra/config/cacert.pem -enddate -noout

   
   Verify the certificate by UI as well:
   
   
   
6. If everything seems to be OK, remove the snapshot.

Another way to address the criticality is to upgrade vRealize Log Insight to new version before the April 30 deadline because the new 8.12 version is not affected by this issue.
At the time I'm posting this article (April 20th, 2023), the new release 8.12 has just been released, and is available for download.
VMware Aria Operations for Logs 8.12 release notes are available here.

That's it.

Log Forwarding - vRealize Log Insight

Issue

I need to forward log from vRealize Log Insight to an external syslog server

Solution

A quick post on how it is simple to configure a log forwarding from vRealize Log Insight to an external syslog server.
The official documentation is available at this link
Below the steps:

1. Get access to the vRealize Log Insight
   
   
2. When logged, click on Administration (1) > Event Forwarding (2) > + NEW DESTINATION (3)
   
3. Fill in the forms, with the proper information like Name, Destination Host. Select the protocol (Ingestion API, syslog, or RAW) to use to send the logs and so on.
   It is also possible to Filter which logs to send adding a filter. In my case I forward everything.
   
4. Click on SAVE
   
   
5. And result should be an entry like the one below:
   

That's it.

How to create an ISO image from a folder on MAC

Issue

How to create an ISO image from a folder on MAC ??

Solution

Lately I have the need to create ISO images to attach as CDROM to the VMs.
I put all stuff I need inside the folder and then I create the ISO image. To do that on MAC is quite easy, you don't need any added software to install. We can use the command "hdiutil".

Open a terminal and type :

hdiutil makehybrid -o image.iso ./folder -iso -joliet 

Replace the 'image.iso' with the name of the iso that you want to create and './folder' with the directory you want to convert into an ISO image.
The folder path can be relative (as in my case) or absolute.

Hit enter.

That's it.

UBUNTU - Extend LVM Partition

Issue

Few months ago I installed Ubuntu on my MAC M1 apple Silicon to practice with a K8s LAB. I proceeded with a simple installation (for beginner) leaving any options set by default, choosing to use the entire disk (30 GB). I made up my configuration, set up minikube, realized some YAML file (just to test some environment in K8s), than weeks later upgraded the kernel .... and now a warnig telling me that I have terminated the space .... what the hell ... how i did?

Watching better the output of the "df -h" command, I realized that the size of the disk wasn't as expected.

Solution

During the initial installation phase I didn't pay to much attention on how the disk was really partitioned.
Let's perform a couple of commands to be sure on the real size of the disk.

lorenzo@ubuntu:~$ sudo lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL

lorenzo@ubuntu:~$ sudo fdisk -l

Verified that we have the original 30GB disk and space not allocated, what we have to do is to extend or resize the LVM partition to the available free space with the lvextend command.
Let's proceed by steps:

1. Locate the partition to extend running the "df -h" command. We have seen that in our case is the root "/" (93% Used).
   
2. With the following command, let's check in the volume group where are the free space:

   root@ubuntu:~# vgdisplay

   
3. Let's run the below lvextend command to extend the file system.

   root@ubuntu:~# lvextend -l +100%FREE /dev/mapper/ubuntu--vg-ubuntu--lv

   
4. With previous lvextend command we have extended the file system to use 100% of the free space availabe, but still the file system is not updated untill we execute the following resize2fs command:

   root@ubuntu:~# resize2fs /dev/mapper/ubuntu--vg-ubuntu--lv

   
5. Last but not least, check that the space has been expanded as desired, performing "df -h" command again.
   

That's it.

How to retrieve the WWPN of a bunch of ESXi hosts using PowerShell

Issue

I need to quickly retrieve the WWNs of a list of ESXi hosts (not managed by a vCenter)

Solution

To solve this, I thought about making a script in powershell that connects to each ESXi hosts presents in "host-list.txt" file and retrieves the necessary information. As outcome, we create a csv file "wwpn-list.csv" with all gathered information.
What I'm assuming here, in my script is that all ESXi hosts have the same username and password (in my case respectively root and VMware1!).
Below is the script:

# get-wwpn.ps1
# 
# Lorenzo Moglie 
# Version: 1.0 (29.12.2022)
#
$user = 'root'
$pswd = 'VMware1!'
$Hosts = Get-Content -Path '.\host-list.txt'
$report = @()
foreach ($host in $Hosts){
	#Write-Host "Host:", $host
	Connect-VIServer -Server $host -User $user -Password $pswd
	$temp = Get-VMhost -Name $host | Get-VMHostHBA -Type FibreChannel | Select VMHost, Device, @{N "WWN";E={"{0:X}" -f $_.PortWorldWideName}} | Sort VMHost,Device
	$wwpn = @{}
	$wwpn.VMHost = $temp.VMHost.Name[0]
	$wwpn.vmhba64 = $temp.WWN[0]
	$wwpn.vmhba65 = $temp.WWN[1]
	$wwn = New-Object -TypeName PSObject -Property $wwpn
	Sreport += Swwn
	Disconnect-VIServer -Confirm:$false
}
$report | Select VMHost, vmhba64, vmhba65 | Sort-Object -Property VMHost | Export-Csv .\wwpn-list.csv

That's it.

Edge VM Present In NSX Inventory Not Present In vCenter

Issue

A customer writes me an email asking for help because two Edge nodes in his NSX-T infrastructure had the following critical error (as shown in the picture below):

"Edge VM Present In NSX Inventory Not Present In vCenter"

Solution

This error message, as we can see from the this link was introduced in version 3.2.1.
The customer has already tried to verify what is indicated in the "Recommended Action" and found that the vm-id is not modified, and the Edge VMs are still in the vCenter Inventory.
Asking if he had made any changes, he replies that the only change made was at the vCenter level to update the expired Machine Cert, and that the certificate was revalidated by NSX-T (indeed the communication between the NSX-T system and the vCenter was showing no errors).
In summary, the Edge VMs were still in inventory, nobody had changed the vm-id, the only thing that had changed was the certificate in vCenter.

The customer fixed it himself on the first attempt, by restarting the cluster appliance the VIP was pointing to. By doing so, when the VIP was switched to another appliance of the NSX Manager cluster, the message resolved itself.

As a second attempt, if the first didn't work, after verifying correctly what is indicated in the "Recommended actions" would be to "Redeploy an NSX Edge VM Appliance" if the edge is no longer connected to NSX Manager; otherwise to replace it inside the Cluster (one by one) as indicated in "Replace an NSX Edge Transport Node Using the NSX Manager UI"

That's it.

MacOS - Running Scripts at Login

Issue

In October 2018 I wrote a short post (in Italian) on how to "Remapping Keys in MacOS".
In the post I also wrote that I would publish a second one on how to make this change permanent. For reasons of time (few time available), work, family, etc. I never managed to write it until I forgot about it. Then thanks to Paolo's comment I remembered that I have not posted it anymore. So, I do it now.

Solution

To solve this issue, and make the fix permanent every time I login into my account I decided to use the LaunchAgent features.
More information about LaunchAgent can be found in "Daemons and Services Programming Guide" and in "Script management with launchd in Terminal on Mac" as well.

Let's see below how to run the script when our user logon.

1. Let's start, creating the folder where to place the script to run. In my case I decide to create a new ".lm_scripts" (hidden folder) under my own directory.
   
2. Create a script similar to the followingg, and place it under the new directory (in my case .lm_scrpts).
   For a complete reading on how to find the various parameters, refer to my original post
   

   #!/bin/bash 
       
    hidutil property --matching '{"ProductID":0x221,"VendorID":0x5ac}' --set '{"UserKeyMapping":[{"HIDKeyboardModifierMappingSrc":0x700000063,"HIDKeyboardModifierMappingDst":0x700000037}]}'

3. Give the script executable permissions
   chmod +x remapping.keys.sh
   
4. Create with your favourite editor a new .plist file. I called it com.remapping.keys.plist
   As shown below
   

   <?xml version="1.0" encoding="UTF-8"?>
   <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
   <plist version="1.0">
       <dict>
           <key>Label</key>
           <string>com.remapping.keys.app</string>
           <key>Program</key>
           <string>/Users/lorenzo/.lm_scripts/remapping.keys.sh</string>
           <key>RunAtLoad</key>
           <true/>
       </dict>
   </plist>

   Note that, the program string above must reflect your user and the path where the script remapping.keys.sh is present.
   
   
5. Then, place the file, in my case com.remapping.keys.plist (just created) under the following directory ~/Library/LaunchAgents/
   
   

Now, at your next login the keys of the keyboard keys will be re-mapped as desired.

That's it.