Switches and Network Architecture

[choucove]

For those of you who have worked with larger networking systems I could use your input! I've got a project to replace some switches and really need to plan this out and have some concerns. Currently they have two 24-port 10/100 SMC Tiger switches which are networked into everything in an absolute mess. I'm concerned to make sure I get the right kind of switches and know what I need to do if I need to configure them to make everything run smoothly.

We are looking to upgrade to all gigabit switches, and I've had recommendations from several people to get HP switches. From what I've used of their more simple switches before I'm very impressed with them, but it's not critical they are HP so long as they are high quality! They have just under 48 total connections currently which means I could put in a single 48-port switch and they'd be fine, but that doesn't allow for much upgradability. However, I know that once you start connecting up multiple switches to one another you're in for a world of hurt with broadcast storms. So, just getting two or three 24-port switches may not be so simple either. Right now the network IP address scheming hasn't really been set up for subnets, but that's something we may want to do since they do have a public network and a private network.

My question is what kind of switches do you recommend for this kind of network? Is it necessary that we need to get all Layer 3 switches, or can we make do with just standard Layer 2 managed (or even perhaps unmanaged) switches and just connect them up a certain way? If I'm connecting two or three 24-port switches together, will I need to configure spanning tree for all these devices? Are there other concerns or things that I need to look out for?

Let me know if you need more information, and thank you for your help in advance!

 

[Tovis]

sounds like a lot of data, is it constantly going back and forth or are some of the computers used on occasion?

 

[choucove]

Most all of the computers are pulling data from the internet continually, and it all is connecting to a single domain controller as well within the network.

 

[SightLine]

With the amount of data you are talking about and potentially many if not most devices will be connecting via gigabit I'd personally not cheap out on the switches. One thing you will start running into on cheaper switches is if multiple machines copying large files will saturate the backplane of the switch. I'm personally fond of Cisco but I also came from a long IT career in a prior life and was a CCIE. I used to manage mutiple 6509 switches connected to a couple hundred smaller switches...

Nowadays though I think most of the top level brands will work fine such as from HP, and even the top line models from the likes of SMC would probably work fine. I'd personally look at a Cisco Catalyst switch myself but some of the others would be fine as well. Just make sure is stackable (expandable) and you should be fine regarding storms, collisions, etc. most any switch of this caliber should be stackable. Problems come in on lower end switches when they are unaware of other switches and have insufficient switching bandwidth to handle high loads. A good stackable switch - when you add another, they essentially become one switch. Also at the numbers of seats I'd probably not worry about stp or subnetting. A decent firewall can easily handle the separation of pub/priv with a lot less hassle. On firewalls I'm partial to both Cisco and Juniper devices...

I'd also say - with 48 seats a secondary domain controller would be very smart.... would not be fun if the pdc goes down.

 

[njshorts]

Most all of the computers are pulling data from the internet continually, and it all is connecting to a single domain controller as well within the network.

in that case, your internet connection is your bottleneck... I'd question if gigabit throughput between switches is the solution. A good staggered group of 12-16 port gigabit switches to a primary 24 port gig switch handling uplink and servers should solve this problem inexpensively and quickly. managed or unmanaged, it'll do the job... just depends on how much control/logging you want. I'd go managed so I can view SNMP/MRTG info and filter traffic based on usage. This plan is based on 12-16 machines with a 1gig uplink as opposed to 24 machines on the same uplink... 16 if the usage is mid-range and a few machines are networked printers, 12 if high usage and all workstations.

If I'm wrong, I'd get a Cisco Catalyst setup... but it's probably overkill- both for data and budget.

edit-afterthought- you could also get a few cisco switches with add-in modules and use 10 gig fibre uplinks between em, but again... probably overkill.

 

[choucove]

The problem we are having right now looking into switches is budget as well. While ideally we would just buy a set of three Layer 3 24-port gigabit switches, that's a very high cost, with the cheapest HP layer 3 switches running $1,000 each.

Right now it sounds like they have a total budget of $1,000 to $1,500 max that they can spend to get their switching infrastructure upgraded. Internet speed will always be the bottleneck, especially out here in the middle of nowhere, where the average DSL package is about 2Mb/s download. They have an internet offering coming soon for 4G with speeds up to 27Mb/s that they will be switching to.

I guess I'm also trying to figure out the best way of physically connecting these switches together as well without causing a bunch of problems. From what I have read, if I connect the switches in a ring manner (so that each switch is connected to each of the other switches) then you're going to get a broadcast storm situation unless you configure spanning tree, correct? If we have three switches, do I have to have a certain type to support building trunks between switches, or just look at something that can do a single 10 gig-e fiber like njshorts pointed out?

 

[njshorts]

hear me out on this, i know it sounds a bit crazy... and really isn't ideal.

with good throughput, 1.5k is a very low budget... no chance at running a catalyst, and even with cisco switches that have 10gig-e crosslinks... you're over your budget.

If it were me (assuming you have 48 active)

switch1- 12 port gig unmanaged.
switch2- 12 port gig unmanaged.
switch3- 12 port gig unmanaged
switch4- 12 port gig unmanaged.
switch5- 12 port gig unmanaged.
switch6- 12-24 port gig managed.

connect each switch to the 24 managed, plus each server. this will spread the load, and you can view overall usage via MRTG. troubleshooting the heaviest of offenders will be tough with unmanaged switches. also, this will give gig-e speed to each switch, or dual/quad if bonded- via the primary switch. a slightly higher budget (unless you find a deal that I can't) would provide for managed 12 ports, which can give you further data for active throughput management.

the concept for this strategy is that each cluster of 10/11 machines shares a gig uplink instead of 23 machines sharing the uplink. this also provides for inexpensive growth, as you've got more than 12 ports leftover, depending on the amount of servers. if they only have these 5 downstream switches, a PDC and an internet connection- this can continue for another 17 switches, or 187 nodes (provided there's still one domain controller with a single NIC, adjust accordingly)... quite a bit of expandability, considering that unmanaged 12 ports are under 100$ each for a good quality switch.

big downside to this is obviously that 11 machines are sharing a single uplink (which, really... the server is limited to it's NICs and the internet connection is limited heavily... so it's not a huge issue, unless you are able to upgrade the internet connection beyond what is currently available).

 

[choucove]

That does sound like a really good plan, and I hadn't really thought of running just a whole bunch of smaller port-density switches.

One reason why I was leaning towards the HP switches is I know their managed switches are much cheaper than comparable CISCO switches. Additionally, the HP switches can be managed using a web based GUI while I don't believe any of the CISCO switches yet incorporate that kind of interface. And finally, CISCO switches have a problem with auto-detecting cross-over or straight through cables where as HP switches automatically detect and can compensate. Not a bit deal, but a nice feature for future work.

I've actually ordered a few of these HP 16-port switches before and really like them, though they are unmanaged switches.

Recently I was looking through and thought that these 24-port managed switches might do the trick for us and they're not a bad price for a managed switch. Do you believe it would be better to find some 16-port managed switches like this instead?

One thing to mention is this company is a non-profit and could get some CISCO switches heavily discounted through e-rate. However, it's strictly CISCO brand that I can find that discounted pricing on right now. HP may also offer discounts, but I don't know if so and what price.

 

[njshorts]

I'm not trying to sway you from HP, personally... I'd use a managed ProCurve 24 port if I were putting this project together (funds depending...)... They're my go-to. Rock solid, reliable, great web access... but may be a bit pricey depending on your outlet. If you're looking for procurve, contact CDW and beat em up on price to try and get down to your budget, using the multiple-switch buy to your advantage.

heck, imho... contact cdw, get an account manager, give them your problem and see what they come up with. their firewall guy saved me from a 2k mistake a few years back.

 

[njshorts]

also, that 24 port you listed looks like a good buy... do you have an MRTG server running at this office, out of curiosity? Maybe a smaller (12-16 port) for the internet connection/printers uplinked to workstations/servers on http://www.google.com/products/cata...a=X&ei=Q5t0TpyHIYWutwe7udHVDA&ved=0CGMQ8wIwAA ? you could still continue the multi-switch procedure with this as they expand, but you're set for now.

 

[SightLine]

Well based on that budget I think NJ might have a good plan for you. Unless of course you can manage some sort of insane discount on Cisco switches (and yes they do have a web interface and have had one for years - just that the hardcore admins like and are used to the console interface). HP's switches are also good - I'd personally not go with a dumb switch. Managed at a minimum.

The HP V1810 you linked http://www.newegg.com/Product/Product.aspx?Item=N82E16833316158 is a nice switch with a nice feature set. QoS traffic shaping is also something you might want at some point in the future - say you change up to an IP based phone service. Very nice to give that traffic priority. Being a semi high end switch HP lists some pretty nice features like QoS, storm prevention, LCAP (port teaming), etc... and they will play nice together when linked to each other.

Also as NJ mentioned - get setup with CDW and get an account manager. The prices on their site - are very much negotiable depending on how much you are spending.

 

[choucove]

That's some good advice for the CDW account as well. I know that the company has already been set up with a CDW account from previous orders of volume licensing, but I just didn't see much in the way of HP switches on their website when I looked initially. I'll have to look again! Also, we don't currently have a MRTG server set up. What all would that require and allow for us?

Just to give you more of an idea of this office here's what is in place. Their internet is currently supplied as a subnet directly from the public school, so it's the CISCO PIX at the school that is the default gateway technically, though we could set up our own router. That ties into a SonicWall TZ-100 firewall in bridged mode. We then have two 24-port 10/100 SMC TigerSwitches with room in telco rack for up to three switches. We have another full rack for servers, which they are phasing out a couple older systems and consolidating into a single server running virtual servers. They also have a second internet connection, just a simple DSL line, that only connects two separate wireless routers which creates a public open wireless network, but that way it's completely separate from the other wired private network.

Space wise, it seems like the most beneficial setup right now is to go with a single Layer 2 managed 48-port gigabit switch connected to a single Layer 2(+) managed 24-port gigabit switch. That will still give them a lot of room to expand but don't have to worry about trunking a bunch of lines together to help alleviate bandwidth issues from one set of switches to the next. If all the primary workstations and the server is connected to the larger switch, and have the internet and printers and WAP connected to the smaller switch, then everything should still have quite balanced capabilities.

I didn't know that CISCO had a web interface on their switches as well, that's really nice! That would be nice as, I do also know some CLI for CISCO switches. Is their interface as nice to work in as the HP switches do you know?

We should qualify for these kind of switches through Techsoup:

Techsoup CISCO Switches

 

[njshorts]

seems pretty straightforward... i really think the 48/24 plan based on priority would be a great solution. you can add smaller switches later if necessary, but I'd assume that's a while off- hopefully a better budget for then... also, procurves hold resale value well (not as well as cisco, but...) so they'll be a good overall investment.

mrtg is a basic technology, but many wrappers exist... and have added useful functionality. if you'd like to get ahold of me, I can set up a remote session so you can view my screen and see my MRTG config, as an example of one of many possible solutions. mine monitors over 250 devices on an old used server that we had laying around... pm me your skype/gtalk name.

 

[njshorts]

...and of course, I just found out a tech ran an update on my monitoring service and broke MTRG. great. there goes my saturday...

in the meantime, http://oss.oetiker.ch/mrtg/

I use OPSView (nagios wrapper) that contains mrtg as well. You can monitor unlimited variables (desktops, servers, services on each, etc), mrtg for throughput, nmis... really awesome piece of software... all within a web-based gui, although you can also use command-line tools and write your own checks in bash.

 

[SightLine]

Their Catalyst web interface is pretty sweet and has eveolved a good bit over the years. Been a few years since I've used one though. You can find some screenshots of it on google.

http://www.google.com/search?hl=en&...m=isch&source=og&sa=N&tab=wi&biw=1920&bih=932


Those 2960 (gigabit models) are very good switches. Pretty much bulletproof reliability. One thing to note though - Cisco loves putting some loud ass fans in their switches...

----snip----

I didn't know that CISCO had a web interface on their switches as well, that's really nice! That would be nice as, I do also know some CLI for CISCO switches. Is their interface as nice to work in as the HP switches do you know?

We should qualify for these kind of switches through Techsoup:

Techsoup CISCO Switches

 

[njshorts]

in line with sightline... those are the prices? seriously? get em! procurves are great, but that's a good deal for cisco- long term hardware at a very discounted price.

 

[choucove]

We just called in about the specially pricing for CISCO equipment and unfortunately they do not qualify for that pricing. In fact, pretty much no non-profit organization does, so not sure the point of that one...

 

[njshorts]

We just called in about the specially pricing for CISCO equipment and unfortunately they do not qualify for that pricing. In fact, pretty much no non-profit organization does, so not sure the point of that one...

go with the procurve setup, monitor it and keep an eye open for expandability.

 

[choucove]

This has all been incredibly helpful, and I'm working to narrow things down then to a select certain couple of switches.

Do you believe there would be any real difference going with one 48-port and one 24-port of the V1910G series of switches:

V1910-24G
V1910-48G

or to go with a single 48-port 2510 series switch and plan to add another switch in the future as budget allows?

HP 2510G-48

Or is there really not enough difference between the series to warrant the cost?

 

[njshorts]

imho, do the set and allow for expandability... otherwise, the client may question why you're doing the job. build with their interests in mind and you'll have a client for life... this will also make your future work for them easier.


also, if you explain why you're using a 48 and a 24, they'll know you've signed on to their expansion and success- always a positive.