VMwareCLI

AUST IT Information Technology for everyone

ICMP issue troubleshooting between esxi guests

Issue recently appeared that 2 guests out of 50 not response to ICMP from another vCenter VMs.

Source and destination on the same proadcast domain stratched accross all vCenters.

All VMs in the same vCenter can communicate with no issues.

Just 2 VM experiencing issues.

MAC address appeared on the source VM.

Checked mac address table on all equipements, looks ok.

Next step to do packetcap on vmnic of destination VM.
NSX Firewall Rules investigation

111
Enable RSS in the ESXi host

To take full advantage of the Intel® Converged Network Adapter's capabilities, enable Receive Side
Scaling (RSS) in the ESXi host to balance the CPU load across multiple cores. Refer to the VMware
performance paper VXLAN Performance Evaluation from VMware 5.1 for performance results
with Intel® Converged Network Adapter X520 with RSS enabled:

ESXi VSwitch | Firewall Performance Testing

Enable pnic RSS / mq:
in shell:
vmkload_mod -u ixgbe
vmkload_mod ixgbe RSS=”4,4,4,4,4,4″

in .vmx or advanced config:
ethernetX.pnicFeatures = “4”

allow multiple vnic thread for single vm:
ethernetX.ctxPerDev = “1”
sources:
http://blogs.vmware.com/performance/2015/04/network-improvements-vsphere-6-boost-performance-40g-nics.html
VMware CLI notes
Some useful NSX CLI commands:

System related:
- ESXi build and version numbers
  esxcli system version get
- Show hostname:
  hostname
- Show host IP address:
  hostname -i
- date and time of when ESXi was installed.
  esxcli system stats installtime get
- command to put ESXi in maintenance mode or take it out
  esxcli system maintenanceMode set –enable true
- Use this command to reboot or shutdown ESXi. The -d parameter is a countdown timer; minimum 10 seconds. ESXi must be in maintenance mode before you can use the command.
  esxcli system shutdown reboot -d 10 -r “Patch Updates”
- To display the current parameters state of particular module:
  esxcli system module parameters list -m ixgben
- check disk space on esxi:
  vdf -h
Storage section:
- The command generates a list of extents for each volume as well as the corresponding device name to UUID mapping
  esxcli storage vmfs extent list
- The command lists of all the volumes or datastores accessible by the ESXi host.
  esxcli storage filesystem list
iSCSI section:
- Print the mappings between vmhba names and /dev names (Print HBA devices with identifying information)
  esxcfg-scsidevs -a
- Print the mappings between vmhba names and /dev names (Print mappings for VMFS volumes to their Service Console partitions and vmhba names.)
  esxcfg-scsidevs -m | wc -l
- check...
  vmkchdev -l
- check...esxcli iscsi software get
- List of iSCSI adaptors, UID and states:
  esxcli software adaptor list
- To verify that the local iSCSI initiator has been able to connect to a certain iSCSI SAN we could filter for the TCP/3260 (default port for iSCSI Target) :
  esxcli network ip connection list | grep 3260
- Configuring monitoring using esxtop to monitor storage performance per HBA:
  esxtop -> f (to modify fields) -> Press b, c, d, e, h, and j to toggle the fields and press Enter. -> s and 2 (to change update time)
esxcfg-mpath -b

esxcli storage core device list -d naa.xxxxxxxxxxxx

esxcli storage vmfs lockmode list

esxcli system settings advanced list --option /VMFS3/HardwareAcceleratedLocking

Network section:
- Show IP addresse for all vmk's:
  esxcfg-vmknic -l
less vobd.log

esxcli network ip connections list | grep 1234
- Show all vmnic's settings:
  esxcfg-nics -l
- Show extended vmnic settings:
  esxcli network nic get -n vmnic1
- Show all vitrual switches
  esxcfg-vswitch -l
- Check vmnic statistic:
  esxcli network nic stats get -n vmnic2
- Displays the IPv4 configuration for all the interfaces present on ESXi
  esxcli network ip interface ipv4 get
Network section (vsish):
- Show statistic on card:
  cd /net/pNics/vmknic5 cat stats
Other sections:
- Search in archived logs (.gz):
  zcat vmkernel.*.gz | less
  zcat vmkernel.0.gz | head
  zcat vmkernel.0.gz | grep -i sence | egrep -v "0x5 0x20|0x4 0x24"
zcat vobd.*.gz | grep -i

vmware -vl

VIB sections:
- Show installed VIBs: esxcli software vib list
- Uninstall VIBs: esxcli software vib remove -n nsx-nsxv
- Install VIBs: esxcli software vib install -v /vmfs/volume/BronseNFS/install/VMware_bootbank_esx-nsxv_6.5.0-0.0.85...vib
- Upgrade VIBs: esxcli software vib updrade -v /vmfs/volume/BronseNFS/install/VMware_bootbank_esx-nsxv_6.5.0-0.0.85...vib
Software sections:
- Show module version:
  vmkload_mod -s bnx2i | grep Version
Packet capture:

pktcap-uw --vmk vmk0 --dir 1 -o /tmp/CAPTURE.pcap
pktcap-uw --uplink vmnic3 --dir 1 -o /tmp/CAPTURE.pcap
pktcap-uw --switchport 50331657 --dir 1 -o /tmp/CAPTURE.pcap

Capture dropped packets:
pktcap-uw --capture Drop --uplink vmnic7 --dir 0 -o /vmfs/volumes/XXX/tmp/CAPdc1vesx008vmnic7_in_d
- Compatability IO driver search:
  vmkchdev -l | grep vmn
  https://www.vmware.com/resources/compatibility/search.php?deviceCategory=io
- View the current ARP table using this command:esxcli network ip neighbor list
- Description: Displays the IPv4 configuration for all the interfaces present on ESXi.
  esxcli network ip interface ipv4 get
NSX related sections:
- To find the correct VIBs use URL:https://NSX_Manager/bin/vdn/nwfabric.properties
Virtual Machine related sections:
- Lists all the running VMs on the hosts. The World ID can be used with other commands to carry out various tasks related to VMs
  esxcli vm process list
VMware PowerCLI start notes

VMware PowerCLI 10.0.0 is the new release of PowerCLI and the first to become multi-platform: it adds support for Mac OS and Linux!

Thanks to Microsoft PowerShell Core 6.0, the open-sourced version of PowerShell that’s available on a number of operating systems, from Windows to Linux to Mac OS.
NSX: Uninstalling stuck in progress
Issue explanation:

I went to prepare a cluster but the hosts never fully installed.
I figured I'd uninstall and give it another try.
Now all the hosts in the cluster report "In Progress" with the cluster status "Uninstalling".
I've rebooted all hosts multiple times and checked that the VIBs have been removed.
```
esxcli software vib list
```
```
esx-vxlan
esx-vsip
esx-dvfilter-switch-security
```
The manager and controllers have been rebooted as well.

Solution:

The vibs had already been uninstalled but the task was still hung.
I was able to work around this issue by disconnecting each host and adding back to a different cluster.
I deleted the cluster that was stuck and was then able to prepare the hosts in a new cluster.
The only think I lost was VM folder structure.
The hosts installed without issue and have now been added into my transport zone.

Note:

1. Do not forget to have DNS server and add there:
- vCenter
- esxi's
- NSX Manager
- Controllers
2. Domain Controller is not compulsory but recommended.

Google AdSence

AUST IT - Computer help out of hours, when you need it most.

Find out why we do it for less.

About

AUST IT will help you resolve any technical support issues you are facing onsite or remotely via remote desktop 24/7. More...

Contacts

Reservoir, Melbourne,
3073, VIC, Australia

Phone: 0422 348 882

This email address is being protected from spambots. You need JavaScript enabled to view it.

Sydney: 0481 837 077

Connect

Join us in social networks to be in touch.

ICMP issue troubleshooting between esxi guests

NSX Firewall Rules investigation

Enable RSS in the ESXi host

VMware CLI notes

Some useful NSX CLI commands:

VMware PowerCLI start notes

NSX: Uninstalling stuck in progress

Popular tag

Who's Online

Google AdSence

AUST IT - Computer help out of hours, when you need it most.

About

Contacts

Connect

Newsletter