How to configure a Nested Virtual SAN Cluster 6.5

How to configure a Virtual SAN cluster for your home lab

1. Install esxi on a physical host. In this case, the server uses the ip 10.1.1.2 (10.1.1.1 had my vCenter appliance). Connect to it using the new host client. Just type the ip or the hostname and log in as root.

2. On the physical host, create two internal standard switches. Do not connect them to an uplink. The first one will eventually be used for vMotion between the nested esxi hosts and the second one for virtual san traffic. Enable Promiscous Mode (critical!!!) on vSwitch1 and vSwitch2.

3. On vSwitch1, create a port group called vMotion. On vSwitch2, create a port group called vsan. This is what my configuration looked like by the time I was finished.

4. Using the host client or the web client), create three virtual machines with 2 vcpus, 6 gbs of memory, three vnics and three disks (4 gbs is not enough for virtual san). Connect the vnics to VM Network, vMotion and vsan. Make the disks 10 gbs, 5gbs and 50gbs. I ended up changing memory to 8gbs though).

5. Once you create the 3 future nested esxi hosts, install them one by one. Do not clone them. Then, proceed to use the dcui and change their hostnames and ip addresses. This is what mine looked like.

6. Connect to your vCenter server using the web client (not the new html client) and create a datacenter and add the three hosts. It should look something like this by the time you finish. Some
of my hosts had ssh enabled; that explains the warnings.

7. Create the vmkernel ports for virtual san and vMotion for the three nested esxi hosts. I used the 10 network for management, the 11 network for vMotion and the 12 network for vsan.  Make sure to test every network with ping once you finish. 

8. Got to your esxi hosts and make the 5gb drive as an SSD drive. 

9. Right click on the datacenter and create a cluster. Name it and enable virtual san. Do not enable anything else for now. Then drag and drop the three hosts. You have the choice of Automatic or Manual (I went with automatic in this case). Otherwise, create the disk groups manually once you finished.

10. Now take a look, if you did it right, the vsanDatastore should be around 150gbs (3 x 50gbs).

Final Note: Here are some captures and commands (including how to log in to the rvc)