Our UNI contingent is presenting at the Enterasys Education Roundtable in Milwaukee today. We’re speaking about the network refresh in the UNI residence halls. The focus is K-series and NAC. Here’s our slide deck.
There was a question recently on the Enterasys mailing list asking how to block all traffic to a particular host (or network) except for a few particular ports. The tech who posed the question was using this service within a default “allow” role. This turns out to be more difficult to implement than it looks at first glance. Of course, it can be solved easily by changing the role from a default “allow” role to a default “deny” role, but that isn’t always practical.
Some background is necessary to understand how policy works. Each type of policy rule has a specific precedence associated with it. A packet that matches multiple rules will follow the rule with the higher precedence. Making the issue more confusing, each type of policy rule also has an attribute id. Attribute ids and precedence are inversely related: the type of rule with the highest precedence – MAC source address – has an attribute id of one.
Below is the policy rule precedence table, taken from the K-series 7.31 Configuration Guide. The rule precedence table in the Policy Manager Guide is quite a bit less detailed (and keep in mind not all switch series can use all rules).
| macsource | Classifies based on MAC source address. | 1 |
| macdest | Classifies based on MAC destination address. | 2 |
| ipxsource | Classifies based on source IPX address. | 3 |
| ipxdest | Classifies based on destination IPX address. | 4 |
| ipxsourcesocket | Classifies based on source IPX socket. | 5 |
| ipxdestsocket | Classifies based on destination IPX socket. | 6 |
| ipxclass | Classifies based on transmission control in IPX. | 7 |
| ipxtype | Classifies based on IPX packet type. | 8 |
| ipsourcesocket | Classifies based on source IP address. | 12 |
| ipdestsocket | Classifies based on destination IP address. | 13 |
| ip frag | Classifies based on IP fragmentation value. | 14 |
| udpsourceportip | Classifies based on UDP source port. | 15 |
| udpdestportip | Classifies based on UDP destination port. | 16 |
| tcpsourceportip | Classifies based on TCP source port. | 17 |
| tcpdestportip | Classifies based on TCP destination port. | 18 |
| icmptype | Classifies based on ICMP type. | 19 |
| iptos | Classifies based on Type of Service field in IP packet. | 21 |
| ipproto | Classifies based on protocol field in IP packet. | 22 |
| ether | Classifies based on type field in Ethernet II packet. | 25 |
| llcDsapSsap | Classifies based on DSAP/SSAP pair in 802.3 type packet. | 26 |
| vlantag | Classifies based on VLAN tag. | 27 |
| tci | Classifies based on Tag Control Information. | 28 |
| port | Classifies based on port-string. | 31 |
A lot of the time, when designing a role, “eyeballing” precedence can be done by asking which type of rule is “more specific”, and also keeping in mind that generally, lower layers on the OSI model have higher precedence.
Let’s return to the original issue but make it more specific. In my network, I’d like to block a set of users from accessing my server 2.2.2.2, which runs several web services open to the world. However, I’d like to allow these users to access web pages on my server (tcp ports 80 and 443) Without remembering the italicized part above, it makes logical sense to create a set of rules like this:
The second rule intuitively feels “more specific,” so on the surface it looks like all traffic should be blocked to 2.2.2.2 except for traffic headed to ports 80 and 443. Unfortunately, that’s incorrect. With the above rules, all traffic to 2.2.2.2 will be dropped.
According to the table above, the first rule has a precedence of 13, and the second two have a precedence of 18. It doesn’t matter that the second two rules have a specific IP listed. The “more specific” part only refers to the rule type, not whether a mask for the subnet is added to it. Traffic destined to port 80 on 2.2.2.2 will match the first two rules, and the top one will take precedence, so the traffic will be dropped.
However, there is a way to deny all traffic to 2.2.2.2 and then “poke holes” through this without resorting to changing the role to deny all traffic. There’s a strange layer 3 rule that actually allows a specification for a layer 4 port. The rule set will look like this:
Now all three of these rules have a precedence of 13, according to the table above, but the second two are more specific. So, all traffic headed towards 2.2.2.2 will be dropped, except traffic to port 80 or 443, which will be allowed.
Take home message: It is possible to block all traffic to a given host except for certain ports without using a default deny role. Use Layer 3 IP socket destination rules rather than layer 4 port destination rules.
Earlier in the week, we said goodbye to our legacy network. This probably should have been heralded by some sort of sending-off ceremony, the blare of trumpets, balloons and perhaps even a parade. Instead, we simply shut off switches as we progressed in upgrading, clearing out the old configurations, and started in with our freshly subnetted network. If a new network could have a similar scent to the “new car smell”, that aroma would be wafted throughout the halls about now.
I’ve got to hand credit to Cabletron- they built one heck of a hardware platform. It’s not often that organizations can get nine or so good years out of network equipment, but we’ve been able to, especially thanks to Enterasys’ support on that great hardware foundation.
By tomorrow, we’ll have checked each and every port in every room in the halls. Thankfully, the findings have been quite positive, with very few issues, but along the way we’ve found jacks that are pushed in or broken, mislabeled patch panels, cables that weren’t fully connected, and switch/network configuration issues. I’m still in awe of all we’ve been able to accomplish in just a bit more than a month’s time.
Our first big test comes this weekend as the RAs move into their rooms. I have confidence we’ll have a great amount of success getting them working and connected, although I do anticipate issues here and there. Having students in every floor of every hall will allow us to get a small picture of performance, stability, Internet demand, etc.
It’s a brave new world we’re embarking on, but we’re ready for it. As we prepare for student training and work out last-minute kinks, I’ll once again say goodbye to our old network gear. Godspeed, old friend!
Perfection, timeliness and stability. These things are often at odds, which is certainly the case with the new network refresh this summer. In the midst of replacing everything, there’s also this expectation that things continue to work. Imagine that!
We have been able to maintain a good level of service for critical devices on the network, but the old axiom “you have to break it before you fix it” seems to be true here. If the level of change wasn’t so great, the big three areas (perfection, timeliness, and stability) wouldn’t be so much at odds, but between new equipment, infrastructure changes (fiber, power, re-cabling and cooling), network segmentation and a new registration system (NAC), we’re wading through a sea of changes.
In the face of all of these changes, I’m again reminded why working at UNI is so great. Our student staff have been flexible and capable in helping in a variety of ways. The Department of Residence staff have helped to overcome obstacles as they’ve come up such as power, cooling and transportation. And Network Services has worked to provide the new fiber and switch configurations needed to bring it all together. For a massive project, it’s being given University-wide support. I’d also like to personally thank our golf cart for helping us get around campus so quickly in the oppressive heat we’ve had recently.
We’ve done a lot right. We’re also ahead of pace. There’s been a few errors, but for the most part we’ve eliminated problems and resolved them as they’ve cropped up. Less than one month to go, and then we’ll be past the finish line. I can’t wait. Onward, ho!
Although DIP switch 8 will reset the admin password, I’ve found taking a few additional precautions useful.
I create an emergency site specific local-only super-user account. The local-only option, a new feature in fw v7 in conjunction with the S release, will bypass radius authentication. This has been an especially useful feature during intermittent network problems where the radius server is partially functional. Also resetting the admin account doesn’t do any good when the radius server is unavailable, so I make that local-only as well.
Excerpt from cli-guide.
set system login local-only {yes | no}
(Optional) Specifies the authentication scope for this user. Valid values: yes, no. yes specifies that authentication is only by way of the local user database even with RADIUS or TACACS+ configured. no specifies that authentication is by way of configured methods.
UPDATE:
This is not to be confused with set authentication login local. Which overrides and effectively disables network management radius realms.
Now that I know about the enterasys cli |find command. I find it very useful. Today I need to tag a new vlan to all the interfaces on our VMware server cluster.
(su)->sh port alias |find vm
Alias on port ge.1.27 set to: vmware
Alias on port ge.1.28 set to: server 1 vmnic1.
Alias on port ge.2.37 set to: server 2 vmnic8.
Alias on port ge.2.40 set to: server 2 vmnic7.
Alias on port ge.3.8 set to: server 4 vmnic4.
Alias on port ge.3.9 set to: server 1 vmnic6.
Alias on port ge.3.12 set to: server 5 vmnic7.
Alias on port ge.3.13 set to: server 4 vmnic8.
Alias on port ge.3.25 set to: server 5 vmnic8.
All our equipment is working really well. I can’t believe we haven’t had any gear arrive DOA. We’ve been running our first K-series building on Enterasys NAC for a week with no problems. Power has been fine. We’re seeing about 5-6A 110v on most partially populated K10s. Heat output and closet cooling is a concern. I haven’t had any units thermal yet. I ran one up to the max operating temp and it was fine.
Jul 13 16:56:10 10.10.x.x System[11]Ambient air temperature is warm (40C).
There are 5 buildings completely re-patched and 4 more to go by Aug 1. There are 5 more buildings yet to have sm fiber terminated.
I brought up OSPF today and configured switches in 4 buildings using the new ospf routed subnets. Once that’s finished I’ll turn on and test NAC there and template to the other waiting buildings. I’m planning to work on PIM, IPV6, OSPFv3, and learning constraints, rmon alerts, and traps next week. The week after we plan to streamline NAC, help devel a
couple new policy profiles and any tie up other loose ends. We’re hoping to have our cat 3 building rewired by mid-August.
Aug 1 we expect people to start trickling in.
Aug 17 students move in in large numbers.
Even Sherlock Holmes would find searching for devices a mystery.
How do most network departments find out where a location connects into a switch port? Consulting some sort of home-grown database/file/printout. What is it like using Enterasys’ NAC to locate a device? Elementary, my dear Mr. Watson.
In the process of replacing all of our equipment, we’re ripping out the old and patching in the new. Doing this, of course, requires us to update documentation pointing us to which location goes to which chassis, line card and port. This will probably be the most time consuming aspect of the new product installation, as getting this right is important for tracking down problems. We’ll do our best to get this right, but here’s a nice fall back with NAC- if you plug in a device on the network, using NAC (or Enterasys’ Compass in Netsight Console) you can search by that MAC address. Or IP address. Or hostname. Or filter down to a certain switch/chassis. Or by the registration information (assuming the device is registered). There are a TON of options to find and filter on using NAC, and over the course of time Enterasys has really refined their fields and searching options to be quite flexible.
If searching is so flexible, why bother to document? I considered this myself today, and a few reasons came to mind. The first is redundancy. It’s just handy to have an extra copy of data somewhere else in case of a failure. The second reason is that using NAC/Console solely for finding devices assumes that the devices are functioning properly. What if a switch fails? I would imagine we could go back through old logs in NAC to locate the last time a device was in a certain location, but if a switch is down and the port is important, that could be pretty useful. I’m sure other reasons exist for or against documentation. If you think of one, could you post it as a comment? It would be interesting to hear others’ ideas and thoughts. We’re also looking for a good program to record our port/jack database, so suggestions are solicited.
I fully anticipate using our sleuthing skills this fall when students arrive. Our new switches plus NAC will help to take the mystery out of our network story at UNI.
I’ve been extremely pleased with our new K10 hardware. We’ve been struggling to install gear as quickly as possible as well as cut-over populated buildings at the same time. So far the K10s have performed flawlessly. I rushed three of them into production and had some help from Enterasys to clear up flaws in my configuration. I had prepared for the worst since this was a brand new product. So far these devices are functioning more like a device that have already been in the field and had a chance to stabilize for a few years. I assume this is due to the close relationship with the S-series. We’ve been running policy and authenticated ports since Wednesday and things appear to be operating very smoothly. I plan to bring three more buildings online next week.
We installed K10s in our first closet yesterday. I had to review the release notes as the linecard commands are new to me. The behavior of the linecard modules is different from other enterasys products I’ve worked with. One a chassis has been booted with a module and the module is removed the configuration is still visible.
cara
Once a card is removed the status shows “Hardware is not physically present.” The port operstatus shows “not-present.” The clear linecard command is used to clear the module config.
->sh port status ge.6.*
Port Alias Oper Admin Speed Duplex Type
(truncated) Status Status (bps)
------------ ---------------- -------- ------- ------ ------- ------------------
ge.6.1 not-pres up 10-t rj45
ge.6.2 not-pres up 10-t rj45
ge.6.3 not-pres up 10-t rj45
...
->sh linecard
Linecard Model Config In Progress
-------- ---------------- ------------------
1 KT2006-0224 false
2 KT2006-0224 false
3 KT2006-0224 false
...
10 false
