How to create a vmfs file system with ESXI 5.0

Starting with 5.0,  a new utility/command called partedUtil is used to partition disks. GPT allows the extent to exceed the 2TB limitation found in previous versions of esxi with older versions of vmfs.

What follows are the instructions on how to partition a lun and create a file system.

Step # 1: Type the command with the -h option to learn about the syntax.
partedUtil -h

Step # 2: Access the directory with the disks and get the attributes of a particular disk
cd /vmfs/devices/disks ; ls
partedUtil get naa.device_name_here

Note: The output should be a set of four numbers. For example: 1044 255 63 16777216
These numbers represent the number of cylinders, the head cound, the number of sectors per track
and the total number of sectors of the device

Step # 3: Partition the disk
partedUtil set naa.device_name_here "1 2048 16700000 251 128"

Note: These numbers represent the partition number (1), the starting sector (2048),
the last sector(16700000), the partition type (251=vmfs) and an attribute (128=bootable).

Step # 4: Create a file system using that first partition
vmkfstools -C vmfs5 -S myfs -b 1M naa.device_name_here:1   <- 1=partition number to use

The new esxi firewall

Esxi introduces a brand new service oriented firewall (not based on iptables).
It can be manipulated with the gui or the cli. It also allows new rules to be added if necessary via the implementation of xml files.

Demo:

# grep firewall /etc/vmware/esx.conf

# more /var/log/esxcli.log

# esxcli network firewall get

# esxcli network firewall ruleset list

# vi /etc/vmware/firewall/fake-ftp.xml
<configroot>
<service>
<id>fake-ftp</id>
<rule id='0000'>
<direction>inbound</direction>
<protocol>tcp</protocol>
<port>21</port>
</rule>
<enabled>false</enabled>
<required>false</required>
</service>
</configroot>

# esxcli network firewall refresh
# esxcli network firewall ruleset list
# esxcli network firewall ruleset rule list | grep fake-ftp

Note:
If you prefer to edit the default file, back it up first and add a sticky bit to it.
# chmod 1777 service.xml

How to Install and Use the Vsphere 5 Syslog Collector

10 Steps to configure the Syslog Collector

01. Using the vcenter server cd, install the syslog collector
02. Connect to your vcenter server and click on Home
03. Click on the new Syslog Collector icon to verify its installation
04. Go to Hosts and Clusters
05. Select your esxi host
06. Go to the Configuration tab
07. Under Software, click on Advanced Settings
08. Click on Syslog, select Remote
09. Input the ip address of the syslog collector (ex: tcp://10.1.1.1:514)
10. Go to home, click on the syslog icon and verify functionality

Using Image Builder

What it can do:

Image Builder allows the administrator to create unique esxi iso images that can contain additional vibs (software packages with extra drivers) or remove some of the default packages if they are not needed. Since additional vibs are usually provided by third party vendors, this demo removes drivers not needed in this particular server. Vibs typically contains a descriptor xml, a digital signature file and the actual package of software with additional drivers.

How to create your own iso image:

Step # 1: Connect to the vcenter server using powercli

set-executionpolicy unrestricted
connect-viserver vcenter.acme.com

Step # 2: Add a software depot and verify the results

add-esxsoftwaredepot C:\Depot\mydepot.zip
get-esxsoftwaredepot

Step # 3: Clone the original iso image and modify the new one by removing the vib called net-bnx2 (for broadcom nics)

get-esximageprofile
new-esximageprofile -cloneprofile "Name of Standard Image from Previous Command" -name myprofile
get-esxsoftwarepackage
remove-esxsoftwarepackage -imageprofile myprofile -softwarepackage net-bnx2

Step # 4: Create a new, modified iso image

export-esximageprofile -imageprofile myprofile -exporttoiso -filepath C:\Depot\myprofile.iso

vscsiStats in action

How to enable vscsiStats for data gathering:

List all running virtual machines and their world numbers.

vscsistats -l

Start gathering statistics for a particular world.

vscsistats -s -t -w 20560

Print output to the console.

vscsistats -p all -w 20560

Redirect output to a file.

vscsiStats -p all -c -w 20560 > /tmp/vmstasts.csv

Stop gathering statistics.

vscsiStats -x -w 20560

Everything about esx/esxi and nfs

Test connectivity between the esx server and nfs server:

# vmkping nfs-server
PING nfs-server (172.16.100.1): 56 data bytes
64 bytes from 172.16.100.1: icmp_seq=0 ttl=253 time=0.171 ms
64 bytes from 172.16.100.1: icmp_seq=1 ttl=253 time=0.171 ms
64 bytes from 172.16.100.1: icmp_seq=2 ttl=253 time=0.168 ms

View if any nfs mounts exist on the esx or esxi server:

# esxcfg-nas -l
ABC is /isos from nfs-server mounted

Unmount an nfs mount:

# esxcfg-nas -d ABC
NAS volume ABC deleted.

Mount the same file system again:

# esxcfg-nas -a ABC -o nfs-server -s /isos -y
Connecting to NAS volume: ABC
ABC created and connected.

Make sure the nfsclient module is loaded:

# vmkload_mod -l | grep nfs
nfsclient           0x418004874000    0x14000     0x417fc55e17c0    0x2000        47 Yes

Unload a load the module if necessary (can't be in use):

# vmkload_mod -u nfsclient
vmkload_mod: Can not remove module nfsclient: module symbols in use

Reload the module:

# vmkload_mod nfsclient
vmkload_mod: Can not load module nfsclient: module is already loaded

Esxcli Examples

View the storage array type plugins available:

# esxcli nmp satp list
Name                 Default PSP    Description
VMW_SATP_ALUA_CX     VMW_PSP_FIXED  Supports EMC CX that use the ALUA protocol
VMW_SATP_SVC         VMW_PSP_FIXED  Supports IBM SVC
VMW_SATP_MSA         VMW_PSP_MRU    Supports HP MSA
VMW_SATP_EQL         VMW_PSP_FIXED  Supports EqualLogic arrays
VMW_SATP_INV         VMW_PSP_FIXED  Supports EMC Invista
VMW_SATP_SYMM        VMW_PSP_FIXED  Supports EMC Symmetrix
VMW_SATP_LSI         VMW_PSP_MRU    Supports LSI and other arrays compatible
                                    with the SIS 6.10 in non-AVT mode
VMW_SATP_EVA         VMW_PSP_FIXED  Supports HP EVA
VMW_SATP_DEFAULT_AP  VMW_PSP_MRU    Supports non-specific active/passive arrays
VMW_SATP_CX          VMW_PSP_MRU    Supports EMC CX that do not use the ALUA
                                    protocol
VMW_SATP_ALUA        VMW_PSP_MRU    Supports non-specific arrays that use the
                                    ALUA protocol
VMW_SATP_DEFAULT_AA  VMW_PSP_FIXED  Supports non-specific active/active arrays
VMW_SATP_LOCAL       VMW_PSP_FIXED  Supports direct attached devices



View the multipathing choices available:

# esxcli nmp psp list
Name           Description
VMW_PSP_MRU    Most Recently Used Path Selection
VMW_PSP_RR     Round Robin Path Selection
VMW_PSP_FIXED  Fixed Path Selection



View the runtime name of a device:

# esxcli nmp psp getconfig -d naa.60060160e09028008e4d328c91a2df11
Current Path=vmhba0:C0:T1:L18

View the multipathing attributes of all devices:

# esxcli nmp device list
naa.60060160e0902800789c7d5d91a2df11
    Device Display Name: DGC Fibre Channel Disk (naa.60060160e0902800789c7d5d91a2df11)
    Storage Array Type: VMW_SATP_CX
    Storage Array Type Device Config: {navireg ipfilter}
    Path Selection Policy: VMW_PSP_MRU
    Path Selection Policy Device Config: Current Path=vmhba0:C0:T1:L4
    Working Paths: vmhba0:C0:T1:L4
...

View if a device is using roundrobin:

# esxcli nmp roundrobin getconfig -d naa.60060160e0902800a6c73dda91a2df11
Errors:
Device naa.60060160e0902800a6c73dda91a2df11 Does not use the Round Robin path selection policy.

View the preferred path of a device:

# esxcli nmp fixed getpreferred -d naa.60060160e0902800ef19e58491a2df11
Errors:
Device naa.60060160e0902800ef19e58491a2df11 Does not use the VMW_PSP_FIXED path selection policy.

Some PowerCLI examples

Note:  All of these examples assume that the script contains the line to
           connect to a esx/esxi host. The line should look like the following:
           Connect-VIServer ip_of_host//hostname -user root -password xxxxx

---

Example # 1:  Power on all of the virtual machines

1. Use notepad to create a script on the windows machine with powercli
    installed. Call the script poweronallvms.ps1

2. Add the following line so that the script contains the following:
    Get-VM | Where-Object {$_.PowerState -eq "PoweredOff"} | Start-Vm

3. Execute the script by typing ./poweronnallvms.ps1 and verify the results
    with the vsphere client or the cli/vcli.

 ---

Example # 2: Create a virtual machine from scratch

1. Create a script called createvm.ps1 with your favorite editor.

2. Add the following lines to the script:
    New-VM -Name Lubuntu -NumCPU 2 -memoryMB 1024 -Datastore \  
    Storage1 DiskStorageFormat Thin -GuestID ubuntuGuest
    New-CDDrive -VM Lubuntu -ISOPath [Storage1]/lubuntu10.10.iso \
    -StartConnected
    Start-VM Lubuntu

3. Execute the script by typing ./createvm.ps1 and verify the results with the
    vsphere client. In this case, the iso image had been copied to the  
    datastore ahead of time.

---

Example # 3: Get generic information about host, vms, storage and network

1. Create a script called getinformation.ps1

2. Add the following lines to the script:
    Get-VMHost
    Get-VM
    Get-Datastore
    Get-DataCenter
    Get-VirtualSwitch
    Get-VirtualPortGroup

3. Execute the script by typing ./getinformation.ps1 and observe the output.

 ---

How to configure iscsi from the cli in 4.1

Steps to follow:

Note: The esxi host uses the ip address 10.1.1.1 for its management network and 10.1.1.2 for iscsi. The iscsi server uses the ip address 10.1.1.3.

1. Create a new standard virtual switch
esxcfg-vswitch -a vSwitch1

2. Create a port group for ip storage
esxcfg-vswitch -A "Ip Storage" vSwitch1

3. Add a network card to the virtual switch
esxcfg-vswitch -L vmnic2 vSwitch1

4. Create a vmkernel port for ip storage with an ip address (10.1.1.2)
esxcfg-vmknic -a -i 10.1.1.2 -n 255.255.255.0 "Ip Storage"

5. Enable the iscsi driver
esxcfg-swiscsi -e

6. Verify that the driver has been enabled
esxcfg-swiscsi -q

7. Find the name of your iscsi hba
esxcfg-scsidevs -a (vmhba33 in this case)

8. Bind the vmkernel port to the port group and verify
esxcli swiscsi nic add -n vmk1 -d vmhba33
esxcli swiscsi nic list -d vmhba33

9. Discover the targets available from a particular server
vmkiscsi-tool -D -a -i 10.1.1.3 vmhba33

10. Use the esxcfg-rescan command to find the new devices
esxcfg-rescan vmhba33

Note:  You can now verify your results by either using the vsphere client or the cli

How to create a vmfs file system with ESXI 4.1

How to create a vmfs file system from the command line

1. After adding a new disk, type esxcfg-scsidevs -c (compressed view) to gather
    information about all the disks and the runtime name of the new device
    In this example, the runtime name was mpx.vmhba1:C0:T1:L0

2. Use the fdisk command to partition the new disk and set the partition type to
    0xfb (vmfs type). In this case, I used the command used was:
    fdisk /dev/disks/mpx.vmhba1:C0T1L0.
    You may need to use commands like "p" to view the  existing partitions, "n" to
    create a new one using 100% of the disk and "t" to specify the type. Once this
    is done, use the "w" command to write/save the changes.

This is what the disk will look like by the time one is finished:

 # fdisk /dev/sdh

The number of cylinders for this disk is set to 10240.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/sdh: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdh1               1       10240    10485744   fb  VMware VMFS

3. Once the disk is partitioned properly, the vmkfstools command is needed. To
    create a new file system and a new datastore, The command used was:

vmkfstools -C vmfs3 -b 1M -S MyDatastore mpx.vmhba1:C0:T1:L0:1
or
vmkfstools -C vmfs3 -b 1M -S Daniel /vmfs/devices/disk/naa.60060160e09028008e4d328c91a2df11:1

Final Note: The -S option was used to specify the datastore name
                   The -b option was used to specify the block size
                   The -C option was used to create a new file system
                   The :1 option used at the end of the command specifies the new slice

How to grow the file system by using a second extent:

Use the vml (symbolic link) of the second disk and add it to the original disk (the one with the
file system). Use the -Z option to add another extent. Don't use the -G option. The -G option
grows the file system size by using the rest of the first disk (whatever is unused).

Obtain the information about the two disks you are using:

cd /vmfs/devices/disks
ls -al | grep -i naa.60060160e09028008e4d328c91a2df11
ls -al | grep -i naa.60060160e0902800ef19e58491a2df11

Add the second extent to the first one:

vmkfstools -Z vml.020011000060060160e0902800ef19e58491a2df11524149442035:1 vml.020012000060060160e09028008e4d328c91a2df11524149442035:1

The power of the vim-cmd command

Despite not having a service console, esxi has a lot of commands available. Want something done from the command line NOT using the esxcfg command? No problem. What follows are examples of the capabilities of the vim-cmd command when a virtual machine is concerned.

 # vim-cmd vmsvc

Commands available under vmsvc/:
acquiremksticket           get.configoption          power.off
connect                          get.datastores            power.on
convert.toTemplate       get.disabledmethods       power.reboot
convert.toVm                 get.environment           power.reset
createdummyvm            get.filelayout            power.shutdown
destroy                           get.guest                 power.suspend
device.connection          get.guestheartbeatStatus  reload
device.connusbdev         get.managedentitystatus   setscreenres
device.disconnusbdev    get.networks              snapshot.create
device.diskadd               get.runtime               snapshot.dumpoption
device.diskaddexisting   get.snapshotinfo          snapshot.get
device.diskremove          get.summary               snapshot.remove
device.getdevices            get.tasklist              snapshot.removeall
device.toolsSyncSet       getallvms                 snapshot.revert
device.vmiadd                gethostconstraints        snapshot.setoption
device.vmiremove          login                     tools.cancelinstall
devices.createnic           logout                    tools.install
get.capability                message                   tools.upgrade
get.config                      power.getstate            unregister
get.config.cpuidmask    power.hibernate           upgrade


List all of the vms registered in that esxi host:

# vim-cmd vmsvc/getallvms

Vmid        Name                            File                          Guest OS      Version       Annotation
32     WinXP-ibex01-a   [SharedVMs] WinXP-ibex01-a/WinXP-ibex01-a.vmx   winXPProGuest   vmx-07    OneSizeXP-R40c.vmdk
64     WinXP-ibex01-c   [SharedVMs] WinXP-ibex01-c/WinXP-ibex01-c.vmx   winXPProGuest   vmx-07    OneSizeXP-R40c.vmdk

How to halt a virtual machine:

# vim-cmd vmsvc/power.off 32

or

# vim-cmd vmsvc/power.shutdown 32

Powering on VM:

# vim-cmd vmsvc/power.on 32

Reboot a VM:

# vim-cmd vmsvc/power.reboot 32
(vim.fault.ToolsUnavailable) {
   dynamicType = <unset>,
   faultCause = (vmodl.MethodFault) null,
   msg = "Cannot complete operation because VMware Tools is not running in this virtual machine.",
}

Get network related information:

# vim-cmd vmsvc/get.networks 32

Networks:
(vim.Network.Summary) {
   dynamicType = <unset>,
   network = 'vim.dvs.DistributedVirtualPortgroup:DVPG-a6 77 07 50 43 f3 38 cd-01 b9 dd 92 d3 d4 e7 b0-dvportgroup-146',
   name = "Production",
   accessible = true,
   ipPoolName = "",
}

Get storage related information about a vm:

# vim-cmd vmsvc/get.datastores 32

name                 SharedVMs
url                  /vmfs/volumes/4d333ff0-28d4682c-49ea-00237da51830
capacity             145492017152
freeSpace            31078744064
accessible           1
type                 VMFS
multipleHostAccess   <unset>

Get snapshot related information about a vm:

# vim-cmd vmsvc/snapshot.get 32

Get Snapshot:
|-ROOT
--Snapshot Name        : 01%2f20%2f2011 15:04:40
--Snapshot Desciption  :
--Snapshot Created On  : 1/20/2011 14:59:11
--Snapshot State       : powered on

Remove all snapshots for that vm:

# vim-cmd vmsvc/snapshot.removeall 32

# vim-cmd vmsvc/snapshot.get 32

Create a snapshot for that vm:

# vim-cmd vmsvc/snapshot.create 32 snappy

# vim-cmd vmsvc/snapshot.get 32

Get Snapshot:
|-ROOT
--Snapshot Name        : snappy
--Snapshot Desciption  :
--Snapshot Created On  : 1/20/2011 20:0:9
--Snapshot State       : powered off

Remove just one snapshot:

# vim-cmd vmsvc/snapshot.remove 32 snappy

How to recover from forgotten root password in esxi

What follows is a procedure to recover from a forgotten root password in ESXI

20 steps to recovery // Non-supported procedure

Note: This was done with a physical server (not nested esxi) & an Ubuntu cd

01. Power off your esxi server. This will obviously not be a clean shutdown.
02. Insert a linux live cd and boot from it (ubuntu linux was used in this case)
03. Once running from the live cd, open up a terminal
04. Type sudo mount /dev/sda5 /mnt (sda was the boot disk in this case)
05. Type ls /mnt and verify that this partition has the state.tgz file
06. Type sudo cp /mnt/state.tgz /tmp
07. Type cd /tmp to change directories to /tmp
08. Type sudo gzip -d state.tgz
09. Type sudo tar xvf state.tar
10. Type sudo gzip -d local.tgz
11. Type sudo tar xvf local.tar
12. Type cd etc to access the directory that contains the shadow file
13. Type sudo vi shadow to edit the file and use the "x" letter to remove the password
14. Type cd /tmp to go back to the /tmp directory
15. Type sudo tar czvf local.tgz etc to recreate the local.tgz file
16. Type sudo tar czvf state.tgz local.tgz to recreate the state.tgz file
17. Type sudo cp state.tgz /mnt to put the file back into its original location
18. Type sudo reboot to reboot the esxi host. Remove the live cd
19. Log into the esxi server
20. Change the root password by typing passwd. This will update the /etc/shadow file.

Note: The shadow file should be edited carefully. Make sure that you remove
          only the password and nothing else. The password resides in the second
          field of the first line. By the time you are done, the line should say root::
          (nothing between the two ":", don't erase the ":"s. The rest of the line will
          have a series of numbers. Leave them alone.

Getting info with the vsish command (esxi only)

The vsish command is used to gather information from an esxi host. It is an extremely powerful utility.

Once the command is typed, the ls command shows the contents of a working directory and the cat
command can be used to see the contents of a file. Finally, the cd command can be used to change
directories.

The first example shows how to get the maximum transfer unit for a particular vmnic.

/bin # vsish
/> cd net
/net/> cd pNics
/net/pNics/> ls
vmnic0/
vmnic1/
vmnic2/
vmnic3/
/net/pNics/> cd vmnic1
/net/pNics/vmnic1/> cat mtu
MTU {
   mtu:1500
}
/net/pNics/vmnic1/> q

The second example shows how to gather cpu related information for a host.

/bin # vsish
/> cd hardware
/hardware/> cd cpu
/hardware/cpu/> ls
packageList/
cpuList/
cpuModelName
cacheInfo
cpuInfo
/hardware/cpu/> cat cpuInfo
CPU global information {
   Hyperthreading state:Hyperthreading state: 3 -> enabled
   HV state:HV state: 3 -> HV Enabled
   Number of packages:2
   Number of cores:8
   Number of CPUs (threads):16
   HV Replay capable:1
   Reason replay is disabled on Host:Reason replay is disabled on Host: 0 -> HV Replay is supported
}
/hardware/cpu/> q