Skip to content

            Lost ?  |  Need an account:
 
ESX NIC Teaming PDF Print E-mail
(31 votes, average 4.55 out of 5)
Written by Tom Hirt   
Tuesday, 10 March 2009 14:02
Article Index
ESX NIC Teaming
Load-balancing Methods
Configure NIC Teaming
Link Aggregation
VLAN Trunking
ESX vSwitch VLAN Configuration
All Pages

 

VMware ESX server offers several different load balancing / NIC teaming options, but which one is right for you? In this article, we will explorer the different NIC teaming options available through the VI administration interface and show you how to configure your ESX server for maximum performance!

Network Architecture Basics


Lets begin with a little background into how VMware infrasture facilites networking. There are 4 basic components that make up the networking architecture: vNIC (virtual network interface card), vSwitch (virtual switch), Physical Network Adapters, and Physical Switches. We will assume you understand the basics of physical network adapters and switches and focus our attention on their virtualized equivalent.

vNIC:

A vNIC is similar to a physical NIC. Each virtual machine has one or more vNIC adapters that it uses to communicate with both the virtual and physical networks. Each vNIC has its own MAC address and can be assigned one or more IP address just like a NIC found in a non virtualized machine.

vSwitch:

A vSwitch is a virtualized version of a layer-2 physical switch. It too acts and responds just like its physical counterpart. However, in VMware we use vSwitches to not only connect our virtual machines to the virtual network, but also for communications with the physical network.

A vSwitch has 2 basic components: Port Groups and Uplink Connections. Port Groups provide connectivity to our vNIC's, its basically what we "plug" our virtual machines into for network connectivity. Uplink Connections serve as the bridge between our virtual and physical network(s). Uplinks allow us to attach our virtual switch(s) to one or more physical network adapters.

VMware Network Overview

It's important to note that a physical NIC can only be attached to 1 vSwitch. In the diagram above, for simplicity, our ESX server is depicted with 4 physical NIC's, all of which are bound to a single vSwitch. However, our ESX server could have included several vSwitches with many more physical NIC adapters that could have had any number of physical interfaces bound to them. In-fact, the only way to perform NIC teaming is by binding 2 or more physical network adapters with a load balancing policy set on the vSwitch, but more about that in a moment.

The following table represents the limits for each of the networking components:

Device Limit
vNIC adapters per virtual machine 4
vSwitch ports per ESX host 4069
vSwitch ports per vSwitch 1016
vSwitches per ESX host 127
Uplinks per vSwitch 32
Uplinks per ESX host 32
vSwitch port groups per ESX host 512
Physical NIC's per host Unlimited

 



Comments
Add New Search
TGL |31/12/2009 15:04:10
Doesn't forcing the channel-group # mode on set up the interface to use PagP and not LACP? I didn't think ESX could utilize PagP.
thirt |04/01/2010 10:21:34
Hi TGL -

Actually, setting the channel-group # mode on forces the ehterchannel to actually ignore both PAgP and LACP. ESX can not use LACP or PAgP.

Thanks for pointing this out though, I have updated the article to reflect this.

-Tom
Steve Swanson |11/01/2010 12:26:56
Great Article. Very useful for a network engineer needing to config the Cisco equipment to talk to the VM. It answered my question very thoroughly. Only thing I noted was that I don't see the portchannel interface allowing the VLAN 99 in this setup. (ie. switchport trunk allowed vlan 99)

Just my .02 but I am not sure if it is required to make this work but is good practice.
Thanks for the article.

Saravanan |18/01/2010 05:16:51
It realy nice one.Any one can understand easily.
yashar fazili  - mr |19/01/2010 14:22:20
:wink:
good job Tom!
my configuration is exactly similar to the one above but let me ask this question.
after configuring the port channel, show spanning-tree will show that the link is truly aggregated and none of including ports are blocked.
however if you unplug one of these links ping test will stop for a sec and then will resume ,which is what being expected !!! but after plunging the unplugged port the ping will fail permanently.
have you faced this problem ? any ideas why is this happening?

regards
yashar fazili

richard marsh  - help with this |27/01/2010 09:56:02
hi tom, thanks for this,...... i have a HP 251048g switch and cannot get this to work by following your info above, im sure HP and cisco do it very differently.... Is there any info you could supply on setting it up with HP....

I have two hosts, both with 2 nics, and i have 2 port groups (one called vlan 3 and vlan4, both with the VID set to 3 and 4 respectively)... on the HP switch, i have tried adding the 4 ESX nics into a trunk... and my vms couldnt see each other, so then i added the 4 nics as tagged on both vlan 3 and vlan 4 on the hp switch.... now vlan 3 vmachines can ping each other across hosts, but still not vlan 4 vmachine...

any help would be great.... i am struggling on this one.... thanks

Rich rich@cisltd.com
hfghgh |28/04/2010 03:41:46
A dude. If I use vswitchs, Do i have to activate somthing in physical switch?

thanks
Styz  - Great job |12/07/2010 14:44:41
Fantastic write up. That's all... just awesome.
NAVEEN KUMAR  - esx seerver |02/08/2010 05:18:37
can u please explain detail flow of packets in esx .
Andy  - IT specialist |06/08/2010 13:31:19
Thank you for a very helpful article!

It works for my environment (IBM BladeCenter HS12 with built-in Cisco CBS3012 and ESXi 4.1) with only couple corrections.
In most practical cases, unfortunately, you should use the same NICs for both VMs and management networks. So when you create or edit your VM network on vSwitch don't forget to tackle your management network too and set up VLAN ID for it. Quite possible you will lose connection in your client, so at that point you need to have console access to ESX host to set up VLAN ID (it should be certain VLAN. "4095" value which suppose to allow all VLANs on network didn't work).
Other complications I got from Cisco switch. I doesn't support any other encapsulation apart from dot1q so you can't issue such command. And in GUI Cisco Nework Assistant which is quite useful tool you can set up for port-channel (EtherChannel) only MAC based load balancing.

Once again thank you for useful guide.
Emre  - Great, thanks. |15/09/2010 00:26:04
Very helpful article. Thank you very much.
Shiraaz  - Physical NIcs on different switches |20/10/2010 10:43:08
Very helpful article thank you.. But which teaming option should I choose in ESX if my vSwitch has physcial NICs that are connected to different switches.. On the physical switch side it would not be possible to configure 802.3ad channels as the ports are not belonging t the same switch
chirag shah  - sr. consultant - IT Infra |11/02/2011 07:06:08
hi,
can you explain how inbound traffice will work in NIC Teaming case for Vsphere 4.1
Bundit S. |25/02/2011 10:50:24
Can you give in deep of usage case of a MAC hash option? I read many paper about it but them just tell me a method but don't give me a usage case and its benefit.

Thanks in advanced.

Wayne |31/05/2011 20:10:08
have a read up on layer 2 and layer 3 network switches. This should hopefully point you in the right direction
bobbydamercer  - 4096 not 4069?? |30/12/2011 15:36:24
Thanks for the great article.


Was just wondering is that typo, it should be '4096' not '4069' ?? :twisted:

TIA
Matt  - Overrun Errors |07/02/2012 09:20:21
Hi Tom,
i have this Config already active on my Cisco 6509 exept for the port-channel load-balance-Command.
On each Interface that is channeled with LAPC, i have incoming overrun errors.

I this load-balance-command a must have?
what does this mean?
Write comment
Name:
Email:
 
Website:
Title:
UBBCode:
[b] [i] [u] [url] [quote] [code] [img] 
 
 
:D:):(:0:shock::confused:8):lol::x:P:oops::cry::evil::twisted::roll::wink::!::?::idea::arrow:
 
Please input the anti-spam code that you can read in the image.

!joomlacomment 4.0 Copyright (C) 2009 Compojoom.com . All rights reserved."

Last Updated on Monday, 04 January 2010 10:16
 

Forum Activity

Author: Cogterrit
May.18.12
Author: Cogterrit
May.18.12
Author: Cogterrit
May.18.12
Author: Cogterrit
May.17.12

Online Stats

Guests Online: 87
Members Online: 1