In this thread, we share lessons the day has taught us.
TIL the importance of having your iSCSI volumes in RAID configuration. My home lab was demolished due to faults within the filesystem within my NAS appliance, which led to several hours of rebuilding server 2012 templates, SQL databases, Active Directory forests, and the such.
Powerful RAID, you have my respect.
(http://vodzilla.co/wp-content/uploads/2013/08/con-air-nic-cage-420x215.jpg)
I only run a single SSD for my iSCSI, but 90% of my hosts are virtual routers and I have the show runs.
For my DC/Vcenter/linux server, I just veeam them monthly and/or clone them to local storage.
Today I learned that my car adapter only puts out 120 watts. That means I don't get to run a heating pad as I drive to/from work.
Oh me achin' back...
Quote from: deanwebb on January 13, 2015, 08:50:18 AM
Today I learned that my car adapter only puts out 120 watts. That means I don't get to run a heating pad as I drive to/from work.
Oh me achin' back...
from one Texan to another: get a truck
Quote from: wintermute000 on January 13, 2015, 04:58:05 AM
I only run a single SSD for my iSCSI, but 90% of my hosts are virtual routers and I have the show runs.
For my DC/Vcenter/linux server, I just veeam them monthly and/or clone them to local storage.
this is a good idea, I'm going to do the same for my CSRs henceforth. my iscsi failed me again this morning, but again it only affected my vmware esxi server; thinking there's something wrong with the iscsi software adapter. failing back to running a 2012 R2 server with Workstation installed, not running vcenter will save me 8GB of RAM which more than makes up for the 2012 install.
Quote from: Seittit on January 13, 2015, 08:59:56 AM
Quote from: deanwebb on January 13, 2015, 08:50:18 AM
Today I learned that my car adapter only puts out 120 watts. That means I don't get to run a heating pad as I drive to/from work.
Oh me achin' back...
from one Texan to another: get a truck
with seat heaters
I need to see if there are any after-market seat mods I can get... I'd like to get a seat massage system, as well.
Wow... learned something new today: kits start at $250 for the three-setting model with massage. http://www.heatyourseat.com/default.aspx
Motorcycle seats are common conversions for heating... So you may want to look that way. Depending how your seats are covered, you may just be able to pop it off and insert the element. You may have a harder time figuring out how to get the juice there vs getting the heat in the seat. Lol.
TIL that Cisco TAC uses GNS3 openly, and will even share their results with customers.
TIL that Cisco's EIGRP is not supported on their IE2000 series IP-Lite feature set. Interestingly enough, it supports OSPF, BGP and....RIP.
Nice work Cisco!
Quote from: Seittit on January 21, 2015, 07:38:08 AM
TIL that Cisco's EIGRP is not supported on their IE2000 series IP-Lite feature set. Interestingly enough, it supports OSPF, BGP and....RIP.
Nice work Cisco!
Wow... now even Cisco customers can complain about Cisco proprietary stuff...
TIL that Fortinet firewalls *want* you to just hop on and start configurin' them. Nice and inviting.
Quote from: deanwebb on January 21, 2015, 07:49:08 AM
Wow... now even Cisco customers can complain about Cisco proprietary stuff...
They are working toward making it an informational RFC.
http://www.ietf.org/archive/id/draft-savage-eigrp-02.txt
Quote from: javentre on January 21, 2015, 10:00:14 AM
They are working toward making it an informational RFC.
http://www.ietf.org/archive/id/draft-savage-eigrp-02.txt
Today I also learned that DUAL is actually Cthulhu
(http://i.imgur.com/aRXsffj.png)
If you like Cthulhu, you'll love Cthulhu Wars. http://www.greeneyegames.com
I love *my* CW game. Oh yes I do. Iä! Iä!
They're leaving out all the good features like stub
TIL - my new ASAx config suffers from the any4 bug....
TILx2 - Polycom phones will work with cisco auto qos:
auto qos voip device cisco-phone
Quote from: LynK on January 21, 2015, 03:52:22 PM
TILx2 - Polycom phones will work with cisco auto qos:
auto qos voip device cisco-phone
That makes sense, as the auto qos macro does the following switchport configurations:
If QoS was not already enabled, enables QoS globally.
•If VLAN-based QoS was configured for the port, reverts to the default port-based QoS (done for all ports on switching modules with 1p1q0t/1p3q1t ports).
•If the port is configured with the switchport command, sets the port trust state to trust CoS.
•If the port is not configured with the switchport command, sets the port trust state to trust DSCP.
•Creates and applies a trust-CoS or trust-DSCP QoS policy to ports on switching modules with non-Gigabit Ethernet 1q4t/2q2t ports, which do not support port trust. I'm not a voice guy (thankfully), but I believe the only layer 2 Cisco-proprietary protocol used in VoIP phone configuration is CDP. Since the macro doesn't enable/disable CDP it should work on any device (even a desktop). I say that, but it may be different between Catalyst switches.
Today I learned what Entropy Labels are...*<|:-{)}
Today I learned how to build an OTV domain. Really awesome stuff, will enable our business to have an disaster recovery solution in the event of another hurricane.
Today I learned, PSTN supports a QoS equivalent
QuoteGETS (Government Emergency Telecommunications Services) is used by government and other designated personnel during crises or emergencies to provide priority processing for local and long distance calls on the public switched telephone network.
Today I learned that there are some features on Cisco gear that consider all addresses to be classful, even if you've got them subnetted.
Plays hell with our network if you consider it to be one big 10.0.0.0/8...
Quote from: mynd on January 22, 2015, 10:02:02 AM
Today I learned, PSTN supports a QoS equivalent
QuoteGETS (Government Emergency Telecommunications Services) is used by government and other designated personnel during crises or emergencies to provide priority processing for local and long distance calls on the public switched telephone network.
you see this in hospitals too. pretty neat eh?
Today I learned that the new cisco 4K branch routers are equal to/less than the G2 ISRs, and they have almost a 10 year life span.
TIL..
This job gives me a natural high.
I had this wonky issue where a phone was pulling an IP but it wouldn't register in call manager. After some troubleshooting I discovered the SVI wouldn't route off site so I looked in the distro and found a duplicate route.
Lesson learned here.. even if it doesn't first appear like it's a routing issue. It might still be a routing issue. Check. Everything.
that a 1% packet loss rate on a 600ms link = 190kb/s maximum individual TCP connection speed due to the laws of physics (without optimisations/WAN acceleration of course)
TIL I learned to not add an EIGRP summary command to the distribution to router link when there is already one on the router to WAN link.
TIL that TFTP still works.
I feel dirty for doing it, but it still works when FTP is failing for whatever reason it's failing for.
show parser macro
details all macro commands sent
TIL how to make a Hitler Rants video... made two related to networking...
https://www.youtube.com/watch?v=YimI1j9BnKU - data center security breach
https://www.youtube.com/watch?v=xodb32luSM0 - problem with the NAC project.
TIL, don't use anything not brand name, made in China on a box you care about.
My Xen box caught fire over the weekend... Quite literally. I was using one of those made in China Molex to SATA power adapters, and this weekend it decided to short. Luckily I was using a name brand PS that cut the power, which killed the flame. I haven't gone through for damage control yet, but likely I just lost the DVD drive.
TIL how to spin up Palo Alto 100 VM in VMware Workstation and integrate it into GNS3.
Sounds worthy of a video tut.
Quote from: Seittit on January 27, 2015, 08:01:15 AM
TIL how to spin up Palo Alto 100 VM in VMware Workstation and integrate it into GNS3.
Sounds worthy of a video tut.
Hey, is there anyway to get our hands on a Demo version of the VM-100? Always liked working with the PA's but not using them at my current gig anymore. And they're too expensive to get lab units
TIL from a friend that when a UPS catches fire, it stays on fire until it's all burned out.
Quote from: SimonV on January 27, 2015, 08:17:10 AM
Hey, is there anyway to get our hands on a Demo version of the VM-100?
Yes, there are two methods that I know of:
- purchase a lab license from CDW (about $600 per year(
- Google for it
TIL how to configure Layer 2 NetFlow on NX-OS.
The fact that I can't apply it per VLAN negates my need though, no desire to see all layer 2 traffic on a 40gig VPC.
Quote from: Seittit on January 27, 2015, 10:27:13 AM
TIL how to configure Layer 2 NetFlow on NX-OS.
The fact that I can't apply it per VLAN negates my need though, no desire to see all layer 2 traffic on a 40gig VPC.
It does sound like it's possible here:
http://www.cisco.com/c/en/us/td/docs/switches/datacenter/sw/4_2/nx-os/system_management/configuration/guide/sm_nx_os_cli/sm_15netflow.html#wp1094178
QuoteConfiguring Layer 2 NetFlow
You can define Layer 2 keys in flexible NetFlow records that you can use to capture flows in Layer 2 interfaces. The Layer 2 keys are as follows:
•Source and destination MAC addresses
•Source VLAN ID
•EtherType from the Ethernet frame
I may be reading this wrong, but it looks like I can't.
Sent from my iPhone using Tapatalk
Quote from: deanwebb on January 27, 2015, 08:22:08 AM
TIL from a friend that when a UPS catches fire, it stays on fire until it's all burned out.
What's up with the fire lately? Was it a hacked UPS with car batteries?
No, it was a straight-up ADC UPS with the last battery popped into place.
TIL that VIRL is licensed on a per install basis.
i.e. you are not technically supposed to run it on your laptop and your home ESXi, even if not @ the same time. I asked them explicitly and they said you have to uninstall one and only use the other.
It phones home, so not really game to push the boundaries.
B@stards.
I wonder if you can spoof it through a internal redirect
Quote from: hizzo3 on January 29, 2015, 08:47:27 PM
I wonder if you can spoof it through a internal redirect
:challenge-considered:
Building DRM into your software: $75,000
Being a Nazi on licensing for educational use: Free
Using the network to enforce DRM policies with a bunch of security minded network professionals that was easily spoofed: LMFAO
Enabling Transparent Page Sharing is basically a deduplication of your RAM on multiple VMs running in ESX. Turning this knob allowed me to run 20 CSR1000v routers (at 2.5 GB RAM) on a 32 GB ESXi whitebox. In fact, they're only consuming 13 GB of RAM total!
To enable the TPS feature, go to Configuration tab > Advanced Settings (under Software) > Mem > Mem.AllocGuestLargePage > Change the value from 1 to 0. - See more at: http://networkjutsu.com/home-lab/ccie-rs-v5-home-lab/#sthash.jYVVnq3k.dpuf
TIL that there is a nasty bug in Cisco wireless controllers that keeps them from doing guest authentication the way we want to do it. They're working on it.
TIL that you shouldn't edit/remove the access-list attached to a route-map applied to a 20GB connection between your data center and the rest of the enterprise without first removing the route-map policy from the interface.
Quote from: sgtcasey on February 05, 2015, 09:09:56 PM
TIL that you shouldn't edit/remove the access-list attached to a route-map applied to a 20GB connection between your data center and the rest of the enterprise without first removing the route-map policy from the interface.
oh I learnt this one the hard way. In my case though, we were tweaking a nested QoS policy on branch routers; our supervisor ridiculed me for being cautious and not pushing out a batch script, but praised me when he saw the router reload due to the bug. Our workaround was to create an additional ACL with our changes applied and make the swap within the nested QoS policy.
Sent from my iPhone using Tapatalk
not to trust another engineers work, especially if it's been 3 years and was 4 engineers ago..... yes, I am finally getting around to finish the 6509 decommission, that nobody else wants to do, the one that was started 3 years ago and never finished.
Quote from: Seittit on January 30, 2015, 02:49:31 PM
Enabling Transparent Page Sharing is basically a deduplication of your RAM on multiple VMs running in ESX. Turning this knob allowed me to run 20 CSR1000v routers (at 2.5 GB RAM) on a 32 GB ESXi whitebox. In fact, they're only consuming 13 GB of RAM total!
To enable the TPS feature, go to Configuration tab > Advanced Settings (under Software) > Mem > Mem.AllocGuestLargePage > Change the value from 1 to 0. - See more at: http://networkjutsu.com/home-lab/ccie-rs-v5-home-lab/#sthash.jYVVnq3k.dpuf (http://networkjutsu.com/home-lab/ccie-rs-v5-home-lab/#sthash.jYVVnq3k.dpuf)
Thanks for that, interesting stuff but after doing my due diligence I have a minor bone to pick with the article
Mem.AllocGuestLargePage Enables backing of guest large pages with host large pages. Reduces TLB misses and improves performance in server workloads that use guest large pages. 0=disable.
1.) TPS is on by default. The guy's article is basically incorrect as far as vmware fundamentals go
2.) The knob you're turning is to disable the TPS behaviour via large memory pages and instead force small pages to be used by the TPS feature THAT IS STILL ACTIVE
http://www.boche.net/blog/index.php/2013/03/19/large-memory-pages-and-shrinking-consolidation-ratios/ (http://www.boche.net/blog/index.php/2013/03/19/large-memory-pages-and-shrinking-consolidation-ratios/)
There is other evidence around the place of people reporting 10-20% better TPS performance with small pages vs large pages so I guess if it works for you then great but the technical description of whats happening is defo not accurate.
I am curious whether you are able to fire up 20 CSRs with that setting = 1, if you followed the same procedure (i.e. one @ a time with a large idle gap to let the memory dedupe).
Also note this incoming change which will probably fly right over those of us who don't update our lab ESXi versions
Update 10/20/14: VMware announced (https://blogs.vmware.com/security/2014/10/transparent-page-sharing-additional-management-capabilities-new-default-settings.html) last week that inter-VM TPS (memory page sharing between VMs, not to be confused with memory page sharing within a single VM) will no longer be enabled by default. This default ESXi configuration change will take place in December 2014.VMware KB Article 2080735 (http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2080735) explains Inter-Virtual Machine TPS will no longer be enabled by default starting with the following releases:ESXi 5.5 Update release – Q1 2015
ESXi 5.1 Update release – Q4 2014
ESXi 5.0 Update release – Q1 2015
The next major version of ESXiAdministrators may revert to the previous behavior if they so wish.
Quote from: wintermute000 on February 07, 2015, 05:30:21 AM
I am curious whether you are able to fire up 20 CSRs with that setting = 1, if you followed the same procedure (i.e. one @ a time with a large idle gap to let the memory dedupe).
The difference was quite dramatic, though I see your point as to the incorrect description.
20 CSRs running with mem.AllocGuestLargePage set to 0
VMs running
(http://i.imgur.com/vnhtpyz.png)
Host resources
(http://i.imgur.com/HGga6aP.png)
20 CSRs running with mem.AllocGuestLargePage set to 1 (default setting)
VMs running
(http://i.imgur.com/gDA3Wr6.png)
Host resources
(http://i.imgur.com/Lvn9Q9h.png)
TIL... Social Engineering ain't just for hackers... sometimes it's for people that need to get stuff done for projects...
cool, I did a quick test with fireflies and noticed almost no difference - TPS reduced it to something hilarious like 4Gb active memory consumed (@ 7x2gb hosts!) with the setting @ 1 or 0, no difference observed, the realtime monitor charts look pretty much identical. Maybe there is a difference with CSRs and not with fireflies for example. Interesting
TIL that if an ASA is missing the same-security-traffic permit inter-interface command, and you try using packet-tracer to run down the issue it will show the traffic being dropped by an ACL, but the ACL name will be blank. I am a little embarrassed on how long it took me to figure out.
TIL that if the configuration guide says "the ports must be configured as trunks" it means they need to be configured as trunks. Doing something in a hurry to get a proof of concept ready, and thinking it does not matter as there isn't a link on those interfaces anyway will come back to bite you in the ass, force you to give up troubleshooting, and start over the configuration from scratch.
Today was not a good day.
-Charles
TIL that project managers that try to do engineering when they need to be making managerial decisions really chap my hide.
TIL that a conditional BGP advertisement route-map can also be used to influence attributes
i.e. neighbor x.x.x.x default-originate route-map BLAH
Everything teaches you to use route-map BLAH to match the routes that need to be in the RiB in order to trigger the default-originate- but today I modelled a niche scenario and viola was able to combine conditional advertisement with AS-path prepending (should work with other normal BGP manipulations).
Interesting side note too discovered via this labbing, an AS path prepend on a neighbor route-map will NOT affect default-originate. Stupid IOS syntax tricks....
Also, the max length of a stack power cable is 1.5m. lol
TIL that Cisco snuck in new hardware requirements within a minor code release of Cisco WAAS. That means that the new code I FTP'd to all my WAAS units in the Gulf of Mexico is not compatible with the hardware, furthermore there is no way to prevent the system from loading the incompatible software upon restart.
Seriously. I need to ask the platforms in the Gulf of Mexico to dislodge their WAAS appliance, fly it back to Houston, where I can perform open heart surgery with a recovery CD.
I love you Cisco, I wish you didn't treat WAAS like a Catelyn Stark treated Jon Snow.
Riverbed have dual images and happily boot off the old image if you load a new dud one lol. Screw waas I have so many issues
TIL that the AC in the lab area switches to heat when it gets cold enough.
TI also L that when the heat blows into the lab area, lots of devices start to beep rather insistently.
Quote from: wintermute000 on February 17, 2015, 01:12:49 PM
Riverbed have dual images and happily boot off the old image if you load a new dud one lol. Screw waas I have so many issues
I hear you, just an estimated $3 million dollars to forklift our WAAS infrastructure for Riverbed. Here's to hope their new Akamai solution is worth sticking around for.
re: WAAS
I'm very please with my Silverpeak units, they've been fairly trouble free and I really push a lot of data through them (multi gbps).
TIL Traceroute uses UDP because in 1988 Router Vendors implemented the RFC's incorrectly
"Traceroute is unique because it enlists the help of both UDP and ICMP. UDP was used because when the tool was first created in 1988 router vendors had misinterpreted the RFC's. The RFC's state that you should never respond to an an ICMP error packet. Router vendors implemented this as "never respond to any ICMP packets." So when traceroute was developed in 1988 UDP had to be used in order to elicit a consistent reply. Today traceroute can be used with the "-I" switch in order to generate Echo-Request packets rather than UDP packets. In this mode it functions identically to Windows tracert."
Source: Quote from GIAC Certified Perimeter Protection Analyst (GPPA) SEC502 Book 502.1 pg 195
I did not know that, cool.
TIL that my PM *still* hasn't called together the big meeting between different groups to resolve the wireless pop-up issue due to us standing up a new RADIUS server... but will still bug out when I say the project is delayed due to that meeting not having happened yet to approve the changes necessary. Not my job to call that meeting, just my job to explain the tech stuff...
Hahahahaha yes UDP traceroutes have been the bane of many junior fw admins (or in my experience what HP deems a 'firewall admin' = person who failed CCENT)
today my coworker discovered not to run a show-tech through the console on a Nexus 7k running several contexts.
the funny factor is that he's doing it all in comic sans.
(meeting ran late so I couldn't post a TIL)
well YIL, had a 2 1/2 hour discussion with Cisco about the 9K's and how ACI works, with demo. barely chipped the iceberg, planning a deep dive next week to get more details. But it looks pretty cool.
TIL that video conferences fall flat when one of the rooms participating doesn't have video gear installed yet.
TI also L that although our voice traffic is marked ef outbound from this location, it can return with a variety of markings on the way back, including "best effort."
TIL no matter how many times you verify with a SP, if their tech shows up without the proper equipment, no circuit will be activated.
TIL never to trust the wireless guys for EIGRP configuration.
Exhibit A:
router eigrp 100
network 10.0.0.0
neighbor 10.255.255.8 Vlan654
neighbor 10.255.255.10 Vlan654
neighbor 10.255.255.12 Vlan654
neighbor 10.255.255.14 Vlan654
neighbor 10.255.255.16 Vlan654
neighbor 10.255.255.18 Vlan654
neighbor 10.255.255.20 Vlan654
neighbor 10.255.255.22 Vlan654
neighbor 10.255.255.24 Vlan654
neighbor 10.255.255.26 Vlan654
neighbor 10.255.255.28 Vlan654
neighbor 10.255.255.30 Vlan654
neighbor 10.255.255.32 Vlan654
neighbor 10.255.255.34 Vlan654
neighbor 10.255.255.36 Vlan654
neighbor 10.255.255.38 Vlan654
neighbor 10.255.255.40 Vlan654
neighbor 10.255.255.42 Vlan654
neighbor 10.255.255.44 Vlan654
neighbor 10.255.255.46 Vlan654
neighbor 10.255.255.48 Vlan654
neighbor 10.255.255.50 Vlan654
neighbor 10.255.255.52 Vlan654
neighbor 10.255.255.54 Vlan654
neighbor 10.255.255.56 Vlan654
neighbor 10.255.255.58 Vlan654
neighbor 10.255.255.60 Vlan654
neighbor 10.255.255.62 Vlan654
neighbor 10.255.255.64 Vlan654
neighbor 10.255.255.66 Vlan654
neighbor 10.255.255.68 Vlan654
neighbor 10.255.255.70 Vlan654
neighbor 10.255.255.72 Vlan654
neighbor 10.255.255.74 Vlan654
neighbor 10.255.255.76 Vlan654
neighbor 10.255.255.78 Vlan654
neighbor 10.255.255.80 Vlan654
neighbor 10.255.255.82 Vlan654
neighbor 10.255.255.84 Vlan654
neighbor 10.255.255.86 Vlan654
neighbor 10.255.255.88 Vlan654
neighbor 10.255.255.90 Vlan654
neighbor 10.255.255.92 Vlan654
neighbor 10.255.255.94 Vlan654
neighbor 10.255.255.96 Vlan654
neighbor 10.255.255.98 Vlan654
neighbor 10.255.255.100 Vlan654
neighbor 10.255.255.102 Vlan654
neighbor 10.255.255.104 Vlan654
neighbor 10.255.255.106 Vlan654
neighbor 10.255.255.108 Vlan654
neighbor 10.255.255.110 Vlan654
neighbor 10.255.255.112 Vlan654
neighbor 10.255.255.114 Vlan654
neighbor 10.255.255.116 Vlan654
neighbor 10.255.255.118 Vlan654
neighbor 10.255.255.120 Vlan654
neighbor 10.255.255.122 Vlan654
neighbor 10.255.255.124 Vlan654
neighbor 10.255.255.126 Vlan654
neighbor 10.255.255.128 Vlan654
neighbor 10.255.255.130 Vlan654
neighbor 10.255.255.132 Vlan654
neighbor 10.255.255.134 Vlan654
neighbor 10.255.255.136 Vlan654
neighbor 10.255.255.138 Vlan654
neighbor 10.255.255.140 Vlan654
neighbor 10.255.255.142 Vlan654
neighbor 10.255.255.144 Vlan654
neighbor 10.255.255.146 Vlan654
neighbor 10.255.255.148 Vlan654
neighbor 10.255.255.150 Vlan654
neighbor 10.255.255.152 Vlan654
neighbor 10.255.255.154 Vlan654
neighbor 10.255.255.156 Vlan654
neighbor 10.255.255.158 Vlan654
neighbor 10.255.255.160 Vlan654
neighbor 10.255.255.162 Vlan654
neighbor 10.255.255.164 Vlan654
neighbor 10.255.255.166 Vlan654
neighbor 10.255.255.168 Vlan654
neighbor 10.255.255.170 Vlan654
neighbor 10.255.255.172 Vlan654
neighbor 10.255.255.174 Vlan654
neighbor 10.255.255.176 Vlan654
neighbor 10.255.255.178 Vlan654
neighbor 10.255.255.180 Vlan654
neighbor 10.255.255.182 Vlan654
neighbor 10.255.255.184 Vlan654
neighbor 10.255.255.186 Vlan654
neighbor 10.255.255.188 Vlan654
neighbor 10.255.255.190 Vlan654
neighbor 10.255.255.192 Vlan654
neighbor 10.255.255.194 Vlan654
neighbor 10.255.255.196 Vlan654
neighbor 10.255.255.198 Vlan654
neighbor 10.255.255.200 Vlan654
neighbor 10.255.255.202 Vlan654
neighbor 10.255.255.204 Vlan654
neighbor 10.255.255.206 Vlan654
neighbor 10.255.255.208 Vlan654
neighbor 10.255.255.210 Vlan654
neighbor 10.255.255.212 Vlan654
neighbor 10.255.255.214 Vlan654
neighbor 10.255.255.216 Vlan654
neighbor 10.255.255.218 Vlan654
neighbor 10.255.255.220 Vlan654
neighbor 10.255.255.222 Vlan654
neighbor 10.255.255.224 Vlan654
neighbor 10.255.255.226 Vlan654
neighbor 10.255.255.228 Vlan654
neighbor 10.255.255.230 Vlan654
neighbor 10.255.255.232 Vlan654
neighbor 10.255.255.234 Vlan654
neighbor 10.255.255.236 Vlan654
neighbor 10.255.255.238 Vlan654
neighbor 10.255.255.240 Vlan654
neighbor 10.255.255.242 Vlan654
neighbor 10.255.255.244 Vlan654
neighbor 10.255.255.246 Vlan654
neighbor 10.255.255.248 Vlan654
neighbor 10.255.255.250 Vlan654
neighbor 10.255.255.252 Vlan654
neighbor 10.255.255.254 Vlan654
passive-interface default
no passive-interface Vlan654
no passive-interface GigabitEthernet0/1
no passive-interface GigabitEthernet0/2
Actual EIGRP peers:
PMPrt1#sh ip eigrp neigh
EIGRP-IPv4 Neighbors for AS(666)
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
2 10.255.255.8 Vl654 12 00:00:30 1 5000 1 0
1 10.255.103.212 Gi0/2 10 23w1d 29 174 0 341843
0 10.255.103.201 Gi0/1 12 23w2d 7 100 0 654733
TIL that certificates are DAMN tricky, a damn sight more tricky than I thought...
Quote from: Seittit on February 19, 2015, 04:32:01 PM
TIL never to trust the wireless guys for EIGRP configuration.
Exhibit A:
router eigrp 100
network 10.0.0.0
neighbor 10.255.255.8 Vlan654
neighbor 10.255.255.10 Vlan654
neighbor 10.255.255.12 Vlan654
neighbor 10.255.255.14 Vlan654
neighbor 10.255.255.16 Vlan654
neighbor 10.255.255.18 Vlan654
neighbor 10.255.255.20 Vlan654
neighbor 10.255.255.22 Vlan654
...
<snip>
WTF ????
TIL even more about certificates, and I solved my problem with them! Well, one of my problems... at least the production wireless will be ready for the new RADIUS back-end come Monday. Guest wireless, not so much right now...
TIL I learned about IXIA network tools, in particular the BreakingPoint application and testing solution, in a nice 2 hours meeting with the IXIA pre-sales team. We should be getting one to demo soon.
TIL how many papers need to be signed to close on a new house.
Wowsers, my right hand feels like it's recovering from a ten hour fap-a-thon
Congratulations on closing your new house! Hopefully your hand recovers before the first of endless projects!
more about Cisco ACI, in a 3 hour deep dive.
+ schwag...
Quote from: config t on January 22, 2015, 08:01:05 PM
TIL..
..even if it doesn't first appear like it's a routing issue. It might still be a routing issue. Check. Everything.
TIL this again ^
TIL that there are times for detailed network diagrams and there are times for very generic diagrams. Know your audience before you fire up the Visio.
I am a third of the way done with the aci book its mkay.
TIL this topic has six pages, so it should be stickied.
TIL that I do way too much. Been at my job almost 7 months. I pulled a report on our ticketing. I have closed more in that short amount of time than the next closest person to me has in the last 2 years. :zomgwtfbbq:
TIL that we need to start a round of meetings with four other departments about how NAC affects their stuff.
NAC means talking to EVERYONE in IT... EVERYONE...
fiberchannel. its like they wanted to be ethernet, but different, and more difficult. Because, just because. (or maybe my instructor's bias. LOL)
TIL..
Dean, our security folks denied it for two days, but it WAS the firewall. :problem?:
Then they should all be fired. They have betrayed the code of the Security Guy.
:notthefirewall:
TIL that TACACS+ will work on some firewalls, but not on others, even though they're configured identically...
TIL that an ISP will sell a customer a 150Mbps circuit when their own provided equipment maxes at 100Mbps. :naughty:
TIL...that IOU doesn't support any L2 QoS, only L3...after I built out the whole lab :doh:
Quote from: Nerm on March 06, 2015, 01:18:19 PM
TIL that an ISP will sell a customer a 150Mbps circuit when their own provided equipment maxes at 100Mbps. :naughty:
I used to work at a small ISP and that was a heated topic. Sales saw fast revenue and that was it for them. Eventually a better pre-sales process was implemented that included capacity planning. They still squeaked some through from time-to-time. :twisted: :angry:
Quote from: routerdork on March 09, 2015, 04:14:01 PM
TIL...that IOU doesn't support any L2 QoS, only L3...after I built out the whole lab :doh:
And that is why it was dropped from the CCIE. Its too platform/ASIC specific for IOU.
Quote from: that1guy15 on March 09, 2015, 04:34:44 PM
Quote from: routerdork on March 09, 2015, 04:14:01 PM
TIL...that IOU doesn't support any L2 QoS, only L3...after I built out the whole lab :doh:
And that is why it was dropped from the CCIE. Its too platform/ASIC specific for IOU.
That's what my QoS instructor said about the inclusion of switches into the training as well. We just did a refresh so I've got some 3560G/E/X's I can get my hands dirty with.
TIL... that if you enable MD5 authentication on a BGP peer that goes through an ASA it will break. Apparently not only does the ASA randomize sequence numbers, but it drops the MD5 option in the TCP packet. Quick modification to the firewall, and everything is working.
-Otanx
TIL That a Watchguard is indeed as horrible as everyone always told me.
(http://www.quickmeme.com/img/4f/4fb334c4ac3340f7b39a4ac6dc6dd0becf191827db4b709e2d24fc24ec9c3ec4.jpg)
What exact fix did you apply otanx? Remove or apply an application inspect rule?
I can't find the exact blog post that I used at work, but the one below is similar. You match bgp with a class map, and then set the options you need.
http://bocloud.blogspot.com/2013/04/special-bgp-configuration-on-asa.html
-Otanx
cheers mate, useful stuff
TIL, when you are configuring redundant connectivity to ESX servers via a vPC, and when the port channels don't come up, but the ports are up, and the interfaces and port channels have the exact same configuration... make sure the ESX servers are in vCenter before you spend more than an hour troubleshooting why the port-channels will not com up no matter what.
:whistle:
TIL that our AD setup is... really... well... the nice way of putting it is, "interesting and full of exciting opportunities!" There's lots of security stuff that interfaces with user directories, so this is not a pleasant thing that IL T. Pleasant or not, though, I gotta make things work...
Quote from: deanwebb on March 20, 2015, 02:20:09 PM
TIL that our AD setup is... really... well... the nice way of putting it is, "interesting and full of exciting opportunities!" There's lots of security stuff that interfaces with user directories, so this is not a pleasant thing that IL T. Pleasant or not, though, I gotta make things work...
I don't think that is specific to your setup. AD is always interesting and full of exciting opportunities.
-Otanx
TIL that whitespace (space) is a valid password character that doesn't show up when doing a show run.
Spent the last week trying to understand why my PPP lab wasn't working when pasting in the given configuration.
Any tips on this one?
No tips on the password thing, other than to not use spaces in the future.
TIL that Java *still* sucks.
TIL that using public IP addresses on the LAN automatically enables 6to4 tunneling on Windows clients, causing all sorts of funky DNS behaviour :)
Wow, and you can't deactivate ipv6 on Windows unless you want disasters to happen. Wow...
Well, you can disable the 6to4 via GPOs which is what we're trying now
Network-minded me thinks you can stop it on the firewall too... Those tunnels go towards a location on the internet.
Teredo is UDP/3544 and IPv6IP is protocol 41. Although GPO is cleaner of course.
TIL that Xbox Live uses Teredo tunneling.
Quote from: Reggle on April 21, 2015, 03:12:59 PM
Network-minded me thinks you can stop it on the firewall too... Those tunnels go towards a location on the internet.
Teredo is UDP/3544 and IPv6IP is protocol 41. Although GPO is cleaner of course.
Yes, found that out too when reading up on it. There was also some Teredo traffic being dropped on our edge firewalls but minimal.
Biggest problem was the clients registering their AAAA record in DNS and that a lot of the client-client and client-server communications were tunneled as 6to4.
It's just a major annoyance for the other teams who expect IPv4 output.
Also interesting is that a client always does a second AAAA query when the 6to4 adapter is enabled.
TIL that I've been filling out my timecard all wrong. :-\
Quote from: SimonV on April 22, 2015, 02:44:53 AM
Quote from: Reggle on April 21, 2015, 03:12:59 PM
Network-minded me thinks you can stop it on the firewall too... Those tunnels go towards a location on the internet.
Teredo is UDP/3544 and IPv6IP is protocol 41. Although GPO is cleaner of course.
Yes, found that out too when reading up on it. There was also some Teredo traffic being dropped on our edge firewalls but minimal.
Biggest problem was the clients registering their AAAA record in DNS and that a lot of the client-client and client-server communications were tunneled as 6to4.
It's just a major annoyance for the other teams who expect IPv4 output.
Also interesting is that a client always does a second AAAA query when the 6to4 adapter is enabled.
I spent at least five mails explaining we are
not disablnig IPv6 but 6to4. Summary of Change Request comes in: Disable IPv6 on all computers. Server guys :doh: Hope they didn't mess up the GPO
TIL how to repair a database table for a webforum. Tapatalk users should be back online now.
that you can do per tunnel QoS on a mGRE DMVPN. eek!
http://www.cisco.com/c/en/us/td/docs/ios/sec_secure_connectivity/configuration/guide/15_0/sec_secure_connectivity_15_0_book/sec_per_tunnel_qos.html (http://www.cisco.com/c/en/us/td/docs/ios/sec_secure_connectivity/configuration/guide/15_0/sec_secure_connectivity_15_0_book/sec_per_tunnel_qos.html)
Also, cisco dcloud 'labs' are 50% sales demo, though being able to type your own show commands beats slideware any day of the week
Quote from: wintermute000 on May 24, 2015, 05:57:14 PM
that you can do per tunnel QoS on a mGRE DMVPN. eek!
http://www.cisco.com/c/en/us/td/docs/ios/sec_secure_connectivity/configuration/guide/15_0/sec_secure_connectivity_15_0_book/sec_per_tunnel_qos.html (http://www.cisco.com/c/en/us/td/docs/ios/sec_secure_connectivity/configuration/guide/15_0/sec_secure_connectivity_15_0_book/sec_per_tunnel_qos.html)
Also, cisco dcloud 'labs' are 50% sales demo, though being able to type your own show commands beats slideware any day of the week
I'm not a WAN guy, but this sounds like something I'd never want to deal with... then again I hate QoS in general.
Quote from: deanwebb on April 28, 2015, 10:36:17 AM
TIL that I've been filling out my timecard all wrong. :-\
Timecard? I thought it was a base requirement to enslave an engineer with a salary! ;)
Quote from: AspiringNetworker on May 25, 2015, 11:21:40 AM
Quote from: deanwebb on April 28, 2015, 10:36:17 AM
TIL that I've been filling out my timecard all wrong. :-\
Timecard? I thought it was a base requirement to enslave an engineer with a salary! ;)
We fill out our time allocation so that the higher-ups can determine if we're allocated properly, or if they need to allocate more resources.
Quote from: AspiringNetworker on May 25, 2015, 11:20:19 AM
Quote from: wintermute000 on May 24, 2015, 05:57:14 PM
that you can do per tunnel QoS on a mGRE DMVPN. eek!
http://www.cisco.com/c/en/us/td/docs/ios/sec_secure_connectivity/configuration/guide/15_0/sec_secure_connectivity_15_0_book/sec_per_tunnel_qos.html (http://www.cisco.com/c/en/us/td/docs/ios/sec_secure_connectivity/configuration/guide/15_0/sec_secure_connectivity_15_0_book/sec_per_tunnel_qos.html)
Also, cisco dcloud 'labs' are 50% sales demo, though being able to type your own show commands beats slideware any day of the week
I'm not a WAN guy, but this sounds like something I'd never want to deal with... then again I hate QoS in general.
No they nailed this correctly! Think about a large distributed network with a large number of sites connecting back to the hub office over a variety of link types. Setting up profiles to match each of those and assigning policies and QoS per each profile. Architect once and deploy everywhere.
So the following profiles:
ATT_MPLS_100Mbps
ATT_MPLS_50Mbps
TW_MPLS_50Mbps
T1_WTF_Do_we_still_have_these
Dial_Up_AYFKM
Quote from: that1guy15 on May 25, 2015, 09:26:12 PM
Quote from: AspiringNetworker on May 25, 2015, 11:20:19 AM
Quote from: wintermute000 on May 24, 2015, 05:57:14 PM
that you can do per tunnel QoS on a mGRE DMVPN. eek!
http://www.cisco.com/c/en/us/td/docs/ios/sec_secure_connectivity/configuration/guide/15_0/sec_secure_connectivity_15_0_book/sec_per_tunnel_qos.html (http://www.cisco.com/c/en/us/td/docs/ios/sec_secure_connectivity/configuration/guide/15_0/sec_secure_connectivity_15_0_book/sec_per_tunnel_qos.html)
Also, cisco dcloud 'labs' are 50% sales demo, though being able to type your own show commands beats slideware any day of the week
I'm not a WAN guy, but this sounds like something I'd never want to deal with... then again I hate QoS in general.
No they nailed this correctly! Think about a large distributed network with a large number of sites connecting back to the hub office over a variety of link types. Setting up profiles to match each of those and assigning policies and QoS per each profile. Architect once and deploy everywhere.
So the following profiles:
ATT_MPLS_100Mbps
ATT_MPLS_50Mbps
TW_MPLS_50Mbps
T1_WTF_Do_we_still_have_these
Dial_Up_AYFKM
Hehe - like I said I'm not a WAN guy (until I have to be), and I hate QoS (until I have to do it - again) so..... guess it's good I work in the DC where I don't worry about this too much. :P
LOL @ "T1_WTF..."
not to get off topic, but T1 is still a widely used, and common infrastructure in today's society. We have about 100 or so sites still on T1 MPLS infrastructure.... Not willing to make the price jump to 10MB metro-e, but also wanting new technology... :drama: :drama: I cant wait until they want video here... haha
Quote from: deanwebb on May 25, 2015, 07:05:06 PM
Quote from: AspiringNetworker on May 25, 2015, 11:21:40 AM
Quote from: deanwebb on April 28, 2015, 10:36:17 AM
TIL that I've been filling out my timecard all wrong. :-\
Timecard? I thought it was a base requirement to enslave an engineer with a salary! ;)
We fill out our time allocation so that the higher-ups can determine if we're allocated properly, or if they need to allocate more resources.
We do the same at my company. This way the business knows which department to bill for our services. I work for a large retail company and IT does not create revenue(even though the business can't run without us) so we have to get our funds from somewhere ;)
Quote from: that1guy15 on May 25, 2015, 09:26:12 PM
T1_WTF_Do_we_still_have_these
Dial_Up_AYFKM
Amen brother!! :banana:
TIL that there is a lot about BGP that I do not know. :glitch:
Quote from: SofaKing on May 26, 2015, 11:18:30 AM
Quote from: deanwebb on May 25, 2015, 07:05:06 PM
Quote from: AspiringNetworker on May 25, 2015, 11:21:40 AM
Quote from: deanwebb on April 28, 2015, 10:36:17 AM
TIL that I've been filling out my timecard all wrong. :-\
Timecard? I thought it was a base requirement to enslave an engineer with a salary! ;)
We fill out our time allocation so that the higher-ups can determine if we're allocated properly, or if they need to allocate more resources.
We do the same at my company. This way the business knows which department to bill for our services. I work for a large retail company and IT does not create revenue(even though the business can't run without us) so we have to get our funds from somewhere ;)
Yeah... I used to do this too as a subcontractor...
TIL how to configure CIMC on a Cisco C220 series server
(note to future self, do not use the same IP for the server and for the CIMC configuration)
TIL that pen testing is about to get a lot more trickier to do properly: https://threatpost.com/security-researchers-sound-off-on-proposed-us-wassenaar-rules/113023
Quote from: ristau5741 on May 27, 2015, 12:12:55 PM
TIL how to configure CIMC on a Cisco C220 series server
(note to future self, do not use the same IP for the server and for the CIMC configuration)
CIMC has saved my bacon more than once.
TIL that when learning the same (external) prefixes in different OSPF processes, the cost is not the differentiator :wall:
do elaborate? I would assume the costs are irrelevant as they fight it out in different OSPF DBs, so it would come down to administrative distance and if equal, route will load balance?
AD was the same for both processes, and it was actually the lowest process number that won. I would have guessed the cost. But we ended up increasing the AD on one process which fixed the issue
http://lostintransit.se/2013/03/15/tiebreakers-with-routes-from-different-ospf-processes/
One for the memory bank
I'm sure it will be on the TS lab!! :)
Quote from: Mowery on June 01, 2015, 12:29:13 PM
Quote from: ristau5741 on May 27, 2015, 12:12:55 PM
TIL how to configure CIMC on a Cisco C220 series server
(note to future self, do not use the same IP for the server and for the CIMC configuration)
CIMC has saved my bacon more than once.
Do you guys know if you can setup the CIMC on a Cisco C240 M3 if it has never been setup without taking down the server?
Quote from: mmcgurty on June 17, 2015, 07:03:22 AM
Quote from: Mowery on June 01, 2015, 12:29:13 PM
Quote from: ristau5741 on May 27, 2015, 12:12:55 PM
TIL how to configure CIMC on a Cisco C220 series server
(note to future self, do not use the same IP for the server and for the CIMC configuration)
CIMC has saved my bacon more than once.
Do you guys know if you can setup the CIMC on a Cisco C240 M3 if it has never been setup without taking down the server?
You will need to reboot the server and enter the CIMC configuration utility to setup CIMC.
Quote from: ristau5741 on June 17, 2015, 07:20:23 AM
Quote from: mmcgurty on June 17, 2015, 07:03:22 AM
Quote from: Mowery on June 01, 2015, 12:29:13 PM
Quote from: ristau5741 on May 27, 2015, 12:12:55 PM
TIL how to configure CIMC on a Cisco C220 series server
(note to future self, do not use the same IP for the server and for the CIMC configuration)
CIMC has saved my bacon more than once.
Do you guys know if you can setup the CIMC on a Cisco C240 M3 if it has never been setup without taking down the server?
You will need to reboot the server and enter the CIMC configuration utility to setup CIMC.
It would be really handy if CIMC would pick up a DHCP address so you could set i up later, in case you forgot. I know I have missed it once or twice, and sometimes taking the server down isn't an option during the day.
TIL that having two VMs with the same MAC address and same IP address can cause problems on the network when trying to reach said IP address.
You don't say :professorcat:
It's the truth, I tell you.
I also learned that even after I provide an IP address, host name, physical location of the VBlock hardware, Remedy ticket number for the initial build, and the day we made it active, the VM guys *still* can't find the VM I requested to be decommed so that we can reclaim the license on that box. :wall:
What scares me most is that we have a directive to place more networking infrastructure into the virtual environment, to cut costs. :glitch:
Quote from: deanwebb on June 17, 2015, 10:40:51 AM
What scares me most is that we have a directive to place more networking infrastructure into the virtual environment, to cut costs. :glitch:
Do not be scared. Virtualization is amazing. The effective removal of all hardware issues due to automated vmotion. Trust me... it is a good thing.
Quote from: LynK on June 17, 2015, 12:39:38 PM
Quote from: deanwebb on June 17, 2015, 10:40:51 AM
What scares me most is that we have a directive to place more networking infrastructure into the virtual environment, to cut costs. :glitch:
Do not be scared. Virtualization is amazing. The effective removal of all hardware issues due to automated vmotion. Trust me... it is a good thing.
Well, we already lost SPAN ports because of virtualization, and when we talk about piping Netflow to those boxes, the data center guys break out in hives. "Our bandwidth! Our bandwidth!" they cry. And then there's the question of how competent the guys in charge of the VMs are, since they shied away from a complicated solution that would allow us to have SPAN ports and, of course, the IP address mess that I mentioned.
But I will grant that I didn't have to set up HA for these boxes. HA is usually a pain, regardless of platform, so that is a good thing.
Quote from: deanwebb on June 17, 2015, 01:16:59 PM
Quote from: LynK on June 17, 2015, 12:39:38 PM
Quote from: deanwebb on June 17, 2015, 10:40:51 AM
What scares me most is that we have a directive to place more networking infrastructure into the virtual environment, to cut costs. :glitch:
Do not be scared. Virtualization is amazing. The effective removal of all hardware issues due to automated vmotion. Trust me... it is a good thing.
Well, we already lost SPAN ports because of virtualization, and when we talk about piping Netflow to those boxes, the data center guys break out in hives. "Our bandwidth! Our bandwidth!" they cry. And then there's the question of how competent the guys in charge of the VMs are, since they shied away from a complicated solution that would allow us to have SPAN ports and, of course, the IP address mess that I mentioned.
But I will grant that I didn't have to set up HA for these boxes. HA is usually a pain, regardless of platform, so that is a good thing.
Why not sFlow? And regarding monitoring your network, what about physical taps and a tap aggregation/network packet broker solution like:
http://www.arista.com/en/solutions/technology-bulletins/585-tap-aggregation (http://www.arista.com/en/solutions/technology-bulletins/585-tap-aggregation)
Vendor is working on a flow importer. And as for tap aggregation... we'd have to pay for those things, right? Maybe some other project will need them, so we won't have to use our budget for that...
Some cool things I learned the other day Studying for the VCP-NV.
1. ESXi 5.5 using a vDS can do netflow. Easy to setup, but no clue how much of a load it would put on a production system.
2. ESXi can SPAN the vSwitch to a VM. Set Host affinity, and setup one VM per hypervisor to do capture of virtual traffic. Have not tested sending the SAPN out a physical NIC.
3. ESXi has a packet capture utility at the command line. The console output sucks, but it can save as a pcap.
-Otanx
Hahaha my VCPN exam is on July fourth. Good luck to you
Quote from: Otanx on June 17, 2015, 04:34:56 PM
2. ESXi can SPAN the vSwitch to a VM. Set Host affinity, and setup one VM per hypervisor to do capture of virtual traffic. Have not tested sending the SAPN out a physical NIC.
Hi Otanx
Can you SPAN an entire VLAN that way? We have an issue with a couple of N5K's because they can only capture ingress or egress for VLANs, not both.
Quote from: wintermute000 on June 17, 2015, 09:13:43 PM
Hahaha my VCPN exam is on July fourth. Good luck to you
Good luck to you as well.
Quote from: SimonV on June 18, 2015, 02:52:47 AM
Can you SPAN an entire VLAN that way? We have an issue with a couple of N5K's because they can only capture ingress or egress for VLANs, not both.
There are other issues with the 5Ks and SPAN. Namely only being able to SPAN at 1G. You can span an entire vSwitch so I would assume so. I have not actually done that in a production network. Wintermute posted this link on another topic, and it has hands on labs for VMWare. Look at the introduction to distributed switch lesson. It has a lab on the SPAN capabilities. That is where I found out about it.
http://labs.hol.vmware.com/HOL/catalogs/
-Otanx
Quote from: Otanx on June 17, 2015, 04:34:56 PM
Some cool things I learned the other day Studying for the VCP-NV.
1. ESXi 5.5 using a vDS can do netflow. Easy to setup, but no clue how much of a load it would put on a production system.
2. ESXi can SPAN the vSwitch to a VM. Set Host affinity, and setup one VM per hypervisor to do capture of virtual traffic. Have not tested sending the SAPN out a physical NIC.
3. ESXi has a packet capture utility at the command line. The console output sucks, but it can save as a pcap.
-Otanx
1.) Use netflow and try to use network i/o control.
2.) Yes this works but it would be hella impossible to get a clean sniff unless you migrated a single vm to said host but it all depends on the issue.
3.) Yeah that works out really well if you do not have a external packet sniffing device then you can save that pcap to a data store and grab it yourself.
Quote from: ristau5741 on June 17, 2015, 07:20:23 AM
Quote from: mmcgurty on June 17, 2015, 07:03:22 AM
Quote from: Mowery on June 01, 2015, 12:29:13 PM
Quote from: ristau5741 on May 27, 2015, 12:12:55 PM
TIL how to configure CIMC on a Cisco C220 series server
(note to future self, do not use the same IP for the server and for the CIMC configuration)
CIMC has saved my bacon more than once.
Do you guys know if you can setup the CIMC on a Cisco C240 M3 if it has never been setup without taking down the server?
You will need to reboot the server and enter the CIMC configuration utility to setup CIMC.
I was afraid that this would be the case from what I was seeing in the documentation. If I hear differently I will let you know.
Quote from: Mowery on June 17, 2015, 08:19:54 AM
Quote from: ristau5741 on June 17, 2015, 07:20:23 AM
Quote from: mmcgurty on June 17, 2015, 07:03:22 AM
Quote from: Mowery on June 01, 2015, 12:29:13 PM
Quote from: ristau5741 on May 27, 2015, 12:12:55 PM
TIL how to configure CIMC on a Cisco C220 series server
(note to future self, do not use the same IP for the server and for the CIMC configuration)
CIMC has saved my bacon more than once.
Do you guys know if you can setup the CIMC on a Cisco C240 M3 if it has never been setup without taking down the server?
You will need to reboot the server and enter the CIMC configuration utility to setup CIMC.
It would be really handy if CIMC would pick up a DHCP address so you could set i up later, in case you forgot. I know I have missed it once or twice, and sometimes taking the server down isn't an option during the day.
Sounds like a feature request! Where are my Cisco SE's at?
Quote from: mmcgurty on June 19, 2015, 07:20:06 AM
Quote from: Mowery on June 17, 2015, 08:19:54 AM
Quote from: ristau5741 on June 17, 2015, 07:20:23 AM
Quote from: mmcgurty on June 17, 2015, 07:03:22 AM
Quote from: Mowery on June 01, 2015, 12:29:13 PM
Quote from: ristau5741 on May 27, 2015, 12:12:55 PM
TIL how to configure CIMC on a Cisco C220 series server
(note to future self, do not use the same IP for the server and for the CIMC configuration)
CIMC has saved my bacon more than once.
Do you guys know if you can setup the CIMC on a Cisco C240 M3 if it has never been setup without taking down the server?
You will need to reboot the server and enter the CIMC configuration utility to setup CIMC.
It would be really handy if CIMC would pick up a DHCP address so you could set i up later, in case you forgot. I know I have missed it once or twice, and sometimes taking the server down isn't an option during the day.
Sounds like a feature request! Where are my Cisco SE's at?
After they've already made the sale?
:partay: :woohoo: :joy: :pub: :cheers:
:problem?:
TIL that VSphere usernames are case sensitive, and that the default account is "Administrator@vsphere.local" not "administrator@vsphere.local". 10 minutes of lab time I won't get back.
-Otanx
TIL that when AD is in a mixed 2003 and 2012 environment, everyone blames NAC when people get kicked off the wireless... because of AD doing weird stuff with reading the user certificates on the mobile devices...
I am learning a lot this week...
TIL that Oracle really needs to stick to databases, and away from networking. A quote from Oracle documentation on networking configurations for Oracle RAC deployment.
"STP events should be contained, either by disabling STP for the VLAN or implementing vendor
specific STP re-convergence optimizations such as Port Fast definitions, RootGuard, BPDU
filtering."
When a database guy reads this his eyes glaze over, and he says "Oh, disable spanning-tree. Oracle says you should." Then I get to read Oracle documentation to figure out what he is talking about.
-Otanx
Quote from: deanwebb on July 08, 2015, 07:56:12 PM
TIL that when AD is in a mixed 2003 and 2012 environment, everyone blames NAC when people get kicked off the wireless... because of AD doing weird stuff with reading the user certificates on the mobile devices...
TIL, you got like 5 days to get off that EOL 2003 OS.... reminds me. I got to rebuild my Dell 1300 Windows 2003 server into something else, but that's another thread.
Quote from: Otanx on July 08, 2015, 10:45:36 PM
A quote from Oracle documentation on networking configurations for Oracle RAC deployment.
"STP events should be contained, either by disabling STP for the VLAN or implementing vendor
specific STP re-convergence optimizations such as Port Fast definitions, RootGuard, BPDU
filtering."
When a database guy reads this his eyes glaze over, and he says "Oh, disable spanning-tree. Oracle says you should." Then I get to read Oracle documentation to figure out what he is talking about.
-Otanx
So when do we get to sit back and watch the network go down like a city at night after a substation trips the entire regional grid? :)
TIL that, by default, all downstream ports on the 3400 are UNI ports and pings are dropped on UNI ports. I had a PC connected to a downstream port. I set up an SVI and tried pinging that but failed. Disabled the firewall on the PC and that didn't work. Tried a different cable, port, PC and it still didn't work. Checked the ARP table on the PC and I can see the switch. Checked the ARP table on the switch and I can see the PC.
:jackie-chan:
Looked up the configuration guide for the switch and found that the switch has a Control Plane Security feature that drops ping on UNI ports by default. :doh:
Oh man, you're going to look back on that and laugh one day.
Today I learnt about this charming Field Notice. Three guesses how.
http://www.cisco.com/c/en/us/support/docs/field-notices/637/fn63743.html
Problem Symptoms
If the suspected Catalyst 6500 supervisor, linecard, or fixed configuration hardware has been in operation for approximately 24 months, the product hardware might fail to boot up due to memory failure during a power cycle event. This is caused by one or more of these actions:
Upgrade the software
Reload the entire product
Reload after installation
Chassis power cycle
Online Insertion Removal/Replacement (OIR)
Note: This issue does not affect boards while the boards are in operation. The board failure might occur after one or more of the actions listed are executed.
Quote from: FilipiNomad on July 17, 2015, 08:47:12 PM
TIL that, by default, all downstream ports on the 3400 are UNI ports and pings are dropped on UNI ports. I had a PC connected to a downstream port. I set up an SVI and tried pinging that but failed. Disabled the firewall on the PC and that didn't work. Tried a different cable, port, PC and it still didn't work. Checked the ARP table on the PC and I can see the switch. Checked the ARP table on the switch and I can see the PC.
:jackie-chan:
Looked up the configuration guide for the switch and found that the switch has a Control Plane Security feature that drops ping on UNI ports by default. :doh:
This is where the RTFM comes in handy. Metro ethernet switches are not normal switches, they are much funkier.
:rtfm:
New Smiley. The code should be obvious.
Quote from: Otanx on July 08, 2015, 10:45:36 PM
I am learning a lot this week...
TIL that Oracle really needs to stick to databases, and away from networking. A quote from Oracle documentation on networking configurations for Oracle RAC deployment.
"STP events should be contained, either by disabling STP for the VLAN or implementing vendor
specific STP re-convergence optimizations such as Port Fast definitions, RootGuard, BPDU
filtering."
When a database guy reads this his eyes glaze over, and he says "Oh, disable spanning-tree. Oracle says you should." Then I get to read Oracle documentation to figure out what he is talking about.
-Otanx
Well duh - no spanning tree - no convergence issues! Win!
Quote from: wintermute000 on July 18, 2015, 03:30:21 AMThis is where the RTFM comes in handy. Metro ethernet switches are not normal switches, they are much funkier.
I've done something similar to a Ciena ME switch and it worked fine, so I didn't think I'd run into any issues. I was actually just helping somebody configure their 3400 and told them it should just take a few minutes. One hour later... :wall:
You'll find that UNI vs NNI is quite common in SP metro ethernet. Its all over the JNCIS-SP material for example - these are Metro Ethernet Foundation standards, not cisco
I'm familiar with UNI vs NNI. Almost all my projects are MetroE and MetroE-over-DWDM using Ciena gear. I've configured a Ciena MetroE switch similarly and it worked fine. The Ciena behaves differently. I didn't realize that Cisco downstream ports were UNI by default, and I haven't read anything on the MEF-CECP study materials about pings being blocked on UNI ports.
TIL that if you have an existing backup config password recoveries on Watchguard XTM's are pretty easy. :cheers:
TIL that if you generate a CA cert then sign a cert for the same box, do NOT throw the IP address of the box into the CN on both certs.... i keep telling work I'm not a security guy :p
TIL how to use Wireshark on the Cisco 3850's. Pretty slick.
Quote from: routerdork on August 11, 2015, 03:29:02 PM
TIL how to use Wireshark on the Cisco 3850's. Pretty slick.
Love this feature. Unless you are past 3.0 and run LAN-BASE. then its a no go. Is one of the reasons I purchased 3850 for my closets and now 90% of them cant run it :(
Reading this https://supportforums.cisco.com/document/12013221/using-3850-embedded-wireshark-wired
It says define a capture filter to reduce load, then it also says it can't take a capture filter
so does it or does it not?
Quote from: wintermute000 on August 12, 2015, 01:33:55 AM
Reading this https://supportforums.cisco.com/document/12013221/using-3850-embedded-wireshark-wired
It says define a capture filter to reduce load, then it also says it can't take a capture filter
so does it or does it not?
I didn't try mine with a filter, I did mine on a whole VLAN that didn't have much traffic. But the doc I read mentioned using an ACL.
Can you try with a filter?
TIL that no matter how many times you change the interface settings on a Palo Alto it will not come up until you hit the commit button. Got the box online with the management interface, then spent about an hour troubleshooting why I could not get E1 to come up. Replaced the cable swapped to E2, changed ports on the switch, etc, etc. Then I saw the little commit button at the top. Oops.
-Otanx
TIL that it is unheard of to reject a salary offer and try to negotiate on an internal position. Lol
TIL that playing loud, aggressive rock and roll right before dialing in is a GREAT way to prep for a meeting.
Especially if I have to deal with network architecture questions.
Before? Put it on mute and keep listening dude! Or at least use a 1 ear bud approach. On a side note, I was in a meeting not long ago that someone did that.
TI also L that...
QuoteIt turns out that when you add another switch to an existing (Cisco 3800/3700) stack, the SNMP agent in the switch stack does not automatically detect the new switch even though the switch management itself does. It is necessary to stop and restart the SNMP indexing for the switch stack to make the switch include the "new" chassis when it re-indexes its list of ports.
switch(config)#no snmp ifmib ifindex persist switch(config)# switch(config)#snmp ifmib ifindex persist
Quote from: hizzo3 on August 20, 2015, 04:24:20 PM
TIL that it is unheard of to reject a salary offer and try to negotiate on an internal position. Lol
Yep, and that is exactly why IT people jump ship to get ahead rather than move up internally in a company. There are exceptions to that rule of course but for the most part yea jumping ship is the only way to get ahead in IT.
Quote from: Nerm on August 21, 2015, 07:08:15 AM
Quote from: hizzo3 on August 20, 2015, 04:24:20 PM
TIL that it is unheard of to reject a salary offer and try to negotiate on an internal position. Lol
Yep, and that is exactly why IT people jump ship to get ahead rather than move up internally in a company. There are exceptions to that rule of course but for the most part yea jumping ship is the only way to get ahead in IT.
Very true... at my last place of employment a guy left the company, came back 6 months later, and got a 20k/year pay raise.
At that same employer, they weren't shocked at all when I turned in my 2-week notice because I was grossly underpaid, though I was appropriately paid for my skillset at the time of hiring - and they admitted that they had no structure to implement pay raises which has been an issue for them to hold on to good people. At that same employer, per company policy, even if I was a junior network tech and moved to "architect of the world", the most I could get was an additional 7% of current pay rate..... >:(
Quote from: AspiringNetworker on August 21, 2015, 11:51:12 AM
the most I could get was an additional 7% of current pay rate..... >:(
That is what I am running into now. Hiring manager agrees with what I've requested is fair... Its a matter of getting HR to approve a large base jump since I'm loosing a large bonus.
Quote from: hizzo3 on August 22, 2015, 09:33:55 AM
Quote from: AspiringNetworker on August 21, 2015, 11:51:12 AM
the most I could get was an additional 7% of current pay rate..... >:(
That is what I am running into now. Hiring manager agrees with what I've requested is fair... Its a matter of getting HR to approve a large base jump since I'm loosing a large bonus.
And if they don't, then brace for them assigning you a ton of documentation projects all of a sudden... "Just in case you win the lottery or something..."
TIL that, according to the Government of India, my first name is *actually* my first and middle names. :problem?:
Re-doing my India business visa application... :-\
TIL that Juniper and Cisco VPNs define proxy-ids differently.
TI also L that, among other things, proxy-ids have to match for a VPN to work.
How did you end up fixing it?
Since the Cisco default is to have a proxy ID for the range being matched for interesting traffic and the Juniper side had been set up with host-by-host proxy IDs, I put a range proxy ID on the Juniper and it all worked just fine.
TIL what dean just posted lol.
Quote from: deanwebb on September 11, 2015, 01:59:56 PM
TIL that Juniper and Cisco VPNs define proxy-ids differently.
TI also L that, among other things, proxy-ids have to match for a VPN to work.
They work differently for practically every vendor. Very annoying.
Quote from: deanwebb on September 11, 2015, 03:01:36 PM
Since the Cisco default is to have a proxy ID for the range being matched for interesting traffic and the Juniper side had been set up with host-by-host proxy IDs, I put a range proxy ID on the Juniper and it all worked just fine.
It depends on the number of subnets you need to cover, but for SRX to ASA/CheckPoint/whatever I tend to go with policy-based by default and manually specify the proxy ID as configured in the security policy. If it's a simple one with just one subnet on each end, a route based usually works fine though, but I still put in manual ProxyIDS :whistle:
However, SRX to SRX sets it to 0.0.0.0/0 which is odd again.
root@Branch-vSRX-01> show security ipsec security-associations index 131074 | match Ident
Local Identity: ipv4_subnet(any:0,[0..7]=0.0.0.0/0)
Remote Identity: ipv4_subnet(any:0,[0..7]=0.0.0.0/0)
When an IPsec request comes in and it's a different vendor:
:frustration:
TIL... NX-OS does not have a reload in or reload at command. Don't f-up your config.
-Otanx
Quote from: Otanx on September 18, 2015, 04:46:40 PM
TIL... NX-OS does not have a reload in or reload at command. Don't f-up your config.
-Otanx
Wow really? This made me go look in a certain vendor config guide to verify it was there ;)
Easter egg on Junos :)
Quotesomeuser@somebox> show version and haiku
Hostname: somebox
Model: srx210he
JUNOS Software Release [12.1R4.7]
New router: 'Type-R.'
Chrome faceplate and neon lights!
Needs a big bat wing.
{primary:node0}
someuser@somebox> show version and haiku
Hostname: somebox
Model: srx210he
JUNOS Software Release [12.1R4.7]
IS-IS screams,
BGP peers are flapping:
I want my mommy!
{primary:node0}
someuser@somebox> show version and haiku
Hostname: somebox
Model: srx210he
JUNOS Software Release [12.1R4.7]
Blessed are the meek:
They shall inherit the earth.
Can I have the moon?
{primary:node0}
someuser@somebox> show version and haiku
Hostname: somebox
Model: srx210he
JUNOS Software Release [12.1R4.7]
Glorious morning
Well beyond what I deserve
Stretch myself and grow
{primary:node0}
someuser@somebox> show version and haiku
Hostname: somebox
Model: srx210he
JUNOS Software Release [12.1R4.7]
Holiday spirit
Christmas comes but once a year
Keep it shining bright
{primary:node0}
someuser@somebox> show version and haiku
Hostname: somebox
Model: srx210he
JUNOS Software Release [12.1R4.7]
Weeks of studying,
Days of lab exercises:
JNCIE.
{primary:node0}
someuser@somebox> show version and haiku
Hostname: somebox
Model: srx210he
JUNOS Software Release [12.1R4.7]
Juniper babies
The next generation starts
Gotta get more sleep
{primary:node0}
someuser@somebox> show version and haiku
Hostname: somebox
Model: srx210he
JUNOS Software Release [12.1R4.7]
An ache you can taste
Be sore as heck tomorrow
One more shot on goal
{primary:node0}
someuser@somebox> show version and haiku
Hostname: somebox
Model: srx210he
JUNOS Software Release [12.1R4.7]
Not just the blue sky
Nor the mountains, nor the sea
Lucky live N.C.
Awesome!
The next time I get access to a Juniper box Im doing this!!
Three more
Quote
My session is dead:
Forgot to commit confirm.
Where are my car keys?
TTL down one
the end nearer with each hop
little packet, poof.
'show version and blame'
Gave away too many names
Now you get haiku
Love the haikus.
TIL that WMI error 0x80041003 is a show-stopper for CounterACT NAC only if we don't deploy the client.
And we're deploying the client.
:greatoffer:
TIL that for an FTP operation to be successful, the FTP server needs to be running the FTP service. :doh: :wall:
Thing is, I'd been going around and around with a backup operator for the last few months on getting this backup job set up, but my device would never connect to the FTP server, at all.
And then the different guy I get from the backup group today to help out says, "Hmm... none of those servers you were working with are running FTP."
And then I was all like...
:phone:
He got me an IP address of an actual FTP server and everything went like a champ.
Quote from: deanwebb on December 03, 2015, 04:38:27 PM
TIL that for an FTP operation to be successful, the FTP server needs to be running the FTP service. :doh: :wall:
Thing is, I'd been going around and around with a backup operator for the last few months on getting this backup job set up, but my device would never connect to the FTP server, at all.
And then the different guy I get from the backup group today to help out says, "Hmm... none of those servers you were working with are running FTP."
And then I was all like...
:phone:
He got me an IP address of an actual FTP server and everything went like a champ.
Wow....
:facepalm2:
TIL the OP has left the building.
Yeah, I think he's working on his CCIE and wants no distractions.
TIL how to get RSS feeds to pipe blogs into the forums. :awesome:
TIL I hate programming with MYSQL.
Quote from: AnthonyC on February 26, 2016, 03:00:01 PM
TIL I hate programming with MYSQL.
I learned that yeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeears ago. :lol:
On that note, there's developers, and then there's that sub-variant of developer known as DBAs.
We need a new animated GIF for them alone. And the even more irritating sub-sub-variant, the MS-SQL DBA who believes that a stretched layer 2 domain is the only possible way of building DR, because that's what his preccccious MS SQL cluster requires. I'm pretty sure they're even worse than *spits* Oracle developers.
Oracle DBAs: :oracle:
MSSQL DBAs: :mssql:
Network admin's reaction after a DBA cons helpdesk into an emergency change request: :whatudo:
Nope, Oracle is the worst. Somewhere in their documentation for 11g it states that you should turn off spanning-tree on the network because the latency introduced will cause the heartbeat to fail. This came up during a deployment meeting during an Oracle RAC deployment. They wanted layer two adjacency, and no spanning-tree.
-Otanx
Turn off spanning-tree?
:no:
TIL (well technically yesterday) that M$ documentation is to never be trusted. Our data storage system is Microsoft's StorSimple which is actually a really neat product IMO. M$ support told us that we needed to do an upgrade to our primary storage system. The support engineers and the upgrade documentation said this was a non-disruptive upgrade that could be ran during normal production hours. They were wrong! I started the upgrade yesterday morning at about 9:30AM. We regained access to the over 25TB's of data on that storage system (with the help of previously mentioned M$ support engineers) at 1:30PM. The upgrade still needs to be done but M$ is trying to figure out what went wrong first before we attempt that again lol.
:whatudo:
I am now of the opinion that there is no upgrade that can be done without disruption during normal business hours.
Zero downtime upgrade of an HA pair? Great. Then we will have zero downtime during the upgrade after normal business hours.
With this new job I have discovered that when you are global with more non-US locations than US locations there isn't really a "after business hours" window. The philosophy here is schedule a maintenance window and just make sure it is announced to everyone. Unfortunately you have days like yesterday where your 1 hour "scheduled" window turns into 4 hours lol.
I get what you are saying though. At my previous job being in a service provider environment upgrades even if "non-disruptive" they were always done after that clients business hours.
Quote from: Nerm on March 08, 2016, 08:37:30 AM
With this new job I have discovered that when you are global with more non-US locations than US locations there isn't really a "after business hours" window. The philosophy here is schedule a maintenance window and just make sure it is announced to everyone. Unfortunately you have days like yesterday where your 1 hour "scheduled" window turns into 4 hours lol.
I get what you are saying though. At my previous job being in a service provider environment upgrades even if "non-disruptive" they were always done after that clients business hours.
Here at Worldwide Global Multinational, we can schedule site work pretty well, since there will be staff at other locations in their normal work day that can do work remotely for a site after its business day. For upgrades that affect the global deployment, when the USA goes home on Friday night, it's time to start the upgrades. The Mideast might get a few bumps on Saturday and Sunday, but as long as everything is working before the guys in the corporate HQ in Europe go online on Monday, it's a good upgrade.
We don't have the luxury of IT staff in all locations. We do have staff in other time zones but they are more setup computers and plug stuff in kind of techs. Our only senior level staff are all located in the US.
Quote from: Nerm on March 08, 2016, 09:16:40 AM
We don't have the luxury of IT staff in all locations. We do have staff in other time zones but they are more setup computers and plug stuff in kind of techs. Our only senior level staff are all located in the US.
We have senior staff on three continents, and are able to follow the sun. By that, I mean that Asia shifts to an 1100-2000 schedule so that they can cover most of the Europe hours. We may pick up a few more support locations, but not with senior IT staff in them.
Quote from: deanwebb on March 08, 2016, 09:44:57 AM
Quote from: Nerm on March 08, 2016, 09:16:40 AM
We don't have the luxury of IT staff in all locations. We do have staff in other time zones but they are more setup computers and plug stuff in kind of techs. Our only senior level staff are all located in the US.
We have senior staff on three continents, and are able to follow the sun. By that, I mean that Asia shifts to an 1100-2000 schedule so that they can cover most of the Europe hours. We may pick up a few more support locations, but not with senior IT staff in them.
Damn! That is awesome.
TIL that deanwebb has it pretty good lol.
I fell bass-ackward into this position, almost, and I enjoy it greatly.
:awesome:
TIL how to configure a VPN tunnel on a Juniper SRX.
Quote from: routerdork on March 08, 2016, 01:14:43 PM
TIL how to configure a VPN tunnel on a Juniper SRX.
So, how was the experience? CLI or web interface?
Quote from: SimonV on March 08, 2016, 01:25:12 PM
Quote from: routerdork on March 08, 2016, 01:14:43 PM
TIL how to configure a VPN tunnel on a Juniper SRX.
So, how was the experience? CLI or web interface?
CLI, I didn't know there was a web interface until my customer sent screenshots of their config. I didn't mind it. I've done some work on the EX series switches in the past so knowing the syntax requirements helped. I can't say that I like the way Juniper is configured (seemed confusing to me) but I also have spent 99% of my tunnel time on ASA/PIX models. So it was a good training.
Who the he77 builds racks and puts rack power supply on/off buttons on the back of the rack, where on a normal rack the door opening handle resides???
Couldn't open the back of the rack, lifted up the little cover to stick my finder in to unlatch the door and well. you can guess the rest. Luckily it was only monitoring gear.
TIL how to read labels, and how to spatchcock a chicken.
TIL that formal policies on incident management mean zero to the group with the emergency, and everything to the group that will resolve that emergency.
I've learned it before, but the lesson is once again fresh in my mind after a device had a kernel panic, dumped memory, and told Vcenter to cancel all its resource reservations.
I could post here every day!
Teaming is an optional install feature on Qlogic Extreme 10gb nic drivers, not installed by default.. Shamefully I had to call support, didn't feel bad when it took them over a hour to figure that out.
TIL I need to amend my 2015 return... after I already got my refund... yeeesh...
TIL how to configure a SPF record in DNS and also how to enable SPF checks in postfix.
Also, it's possible to coincidentally power off your VPS RIGHT BEFORE your provider suffers an unscheduled outage to their global control panel, so you can't even get back in to turn it back on. interestingly this implies a single point of failure for management despite how many availability zones you scale out **cough digitalocean cough**
Because of wintermute TIL about SPF, and how to deploy it (I even read the entire RFC). Now I have to decide if I want to bring this up at an engineering meeting next week.
-Otanx
Since turning SPF checking on my spam rate has gone from 4-5 a day to 1-2.
Spamassassin tags most of the 1-2 so problem is mostly solved :)
I daresay one of our fellow members who just happens to have a truckload of Exchange XP from the noughties knows a lot more about this than me...
Ayup. You want that there SPF turned on. :)
That way, you don't get spam from spoofed addresses, just from offshore spam servers, and those tend to be well-known.
how to get a prescription for adderall :eek: :eek: :eek:
TIL how to do a snoop command on a Juniper SSG firewall.
http://movingpackets.net/2015/01/26/netscreen-packet-capture-snoop/
Fun stuff, especially since it helps me prove...
:notthefirewall:
Does that work for transit traffic, or only for packets directed at the FW?
using it for transit traffic right now
Today I learned how to do admin password recovery for a few of my ACS appliances using CIMC. Next I can remember how to upgrade them.
How to edit the known_hosts file on a Linux box.
http://www.linkedin.com/redir/redirect?url=http%3A%2F%2Fwww%2Ethegeekstuff%2Ecom%2F2010%2F04%2Fhow-to-fix-offending-key-in-sshknown_hosts-file&urlhash=L9UF&_t=tracking_anet
If you change a certificate on a device and have other devices set to strict checking on certificates, the strict box will refuse a connection to the box with the new cert unless one clears the appropriate line in known_hosts. Copying, editing, pasting, and then running the perl command did the trick and I was able to connect to the boxes with new certs once again.
TIL one can get into the FEX operating system and manipulate the FEX from within (with a limited about of functionality)
5K-SWITCH# attach fex 102
Attaching to FEX 102 ...
To exit type 'exit', to abort type '$.'
fex-102# ?
begin Begin with line that matches
clear Reset functions
configure Enter configuration mode
count Count the number of lines in the output
dbgexec Execute debug shell command
debug Debugging functions
diff-clean Remove temp files created by '| diff' filters
echo Echo argument back to screen (useful for scripts)
exclude Exclude lines that match
include Include lines that match
last Display last few lines of the output
mping Run mping
no Negate a command or set its defaults
python Source using python script
reload Reboot FEX
scripting Configure scripting parameters
show Show running system information
shutdown Shutdown FEX
sleep Sleep for the specified number of seconds
system System management commands
tar Archiving operations
tclsh Source tclsh script
terminal Set terminal line parameters
test Test commands
end Go to exec mode
exit Exit from command interpreter
pop Pop mode from stack or restore from name
push Push current mode to stack or save it under name
where Shows the cli context you are in
fex-102#
Cisco had to troubleshoot weird dropping packet issue, determined that there was congestion on the links between the FEX and the 5K
TIL that trying to get something to work with an unsupported IOS version is a lot harder than upgrading it to a supported IOS version.
We are now checking versions before we start changing the configs.
Quote from: ristau5741 on July 27, 2016, 11:45:09 AM
TIL one can get into the FEX operating system and manipulate the FEX from within (with a limited about of functionality)
5K-SWITCH# attach fex 102
Attaching to FEX 102 ...
To exit type 'exit', to abort type '$.'
fex-102# ?
begin Begin with line that matches
clear Reset functions
configure Enter configuration mode
count Count the number of lines in the output
dbgexec Execute debug shell command
debug Debugging functions
diff-clean Remove temp files created by '| diff' filters
echo Echo argument back to screen (useful for scripts)
exclude Exclude lines that match
include Include lines that match
last Display last few lines of the output
mping Run mping
no Negate a command or set its defaults
python Source using python script
reload Reboot FEX
scripting Configure scripting parameters
show Show running system information
shutdown Shutdown FEX
sleep Sleep for the specified number of seconds
system System management commands
tar Archiving operations
tclsh Source tclsh script
terminal Set terminal line parameters
test Test commands
end Go to exec mode
exit Exit from command interpreter
pop Pop mode from stack or restore from name
push Push current mode to stack or save it under name
where Shows the cli context you are in
fex-102#
Cisco had to troubleshoot weird dropping packet issue, determined that there was congestion on the links between the FEX and the 5K
I've heard about issues with this in the past and hidden commands - but mehhhh I think it was more about hidden counters. Interesting.
yup, yup.
At the end of the day Cisco 2Ks are just switches that shift control plane up. Pull the curtans back and you have ZTP and control all handled by a high priced 5K.
Wait, isnt this what SDN and central controllers are trying to do? yes. yes it is.
Don't forget no local switching [emoji14] I was at a juniper presentation about vcf and presenter had a big talking point up "not a fex does local switching" LOL
Quote from: wintermute000 on July 28, 2016, 05:10:24 PM
Don't forget no local switching [emoji14] I was at a juniper presentation about vcf and presenter had a big talking point up "not a fex does local switching" LOL
I've heard that the 2300 Series FEX can switch frames without sending up to the 5K, I didn't see anything in the date sheet I took a quick look at here, but I read it somewhere, don't remember where.
http://www.cisco.com/c/en/us/products/collateral/switches/nexus-2000-series-fabric-extenders/datasheet-c78-731663.html
TIL..
MPLS, L3VPN, MPBGP.
Mind = blown.
Listening to Macklemore & Ryan Lewis - Thrift Shop.
Poppin' tags.. this is f&%ing :awesome:
TIL how to say "hello" in Malagasy: Manahoana.
TIL about DevOps security and Docker container security.
DevOps Protip: security changes can be put in as fast as other changes, everyone has to learn how to deal with it. Also, rapidly changing code means less fun for hackers that like to sit in one place that never changes.
Docker Protip: DO NOT RUN A CONTAINER WITH ROOT PRIVILEGES. EVER.
TIL how to verify the status of a CCIE.
Get the number here: http://www.cciehof.com/
Then log in with your CCO here: https://ccie.cloudapps.cisco.com/CCIE/Schedule_Lab/CCIEOnline/jsp/VerifyCCIE_Form.jsp
The guy I just checked is valid and re-certified. 8)
Quote from: deanwebb on August 10, 2016, 01:53:39 PM
TIL how to verify the status of a CCIE.
Get the number here: http://www.cciehof.com/
Then log in with your CCO here: https://ccie.cloudapps.cisco.com/CCIE/Schedule_Lab/CCIEOnline/jsp/VerifyCCIE_Form.jsp
The guy I just checked is valid and re-certified. 8)
or he can log into the cert tracker and click a few buttons and have the proof emailed to you...
Well, we'll check again after his anniversary date expires.
that website check is the SOP when parsing for hires. Faster than asking the guy to provide evidence....
TIL how to sudo
:badass:
TIL that Sonos audio players participate in STP and build wireless links between devices :eek:
https://en.community.sonos.com/troubleshooting-228999/sonos-and-the-spanning-tree-protocol-16973
Quote from: SimonV on February 08, 2017, 04:39:38 AM
TIL that Sonos audio players participate in STP and build wireless links between devices :eek:
https://en.community.sonos.com/troubleshooting-228999/sonos-and-the-spanning-tree-protocol-16973
So, in spite of the fact that BPDU guard is used for some very good reasons on switch ports, these guys want us to turn it off so we can all listen to the music together.
Why couldn't they come up with a proprietary solution that doesn't break BPDU guard?
Indeed, had some issues at a company where they were installing these. First with BPDUguard, then with the link costs after removing BPDUguard. Wouldn't have expected this on a 'consumer' product but thank god it's just standard STP so it wasn't too hard to get it contained. But imagine what would happen if two of these speakers are installed in different VLANS and then team up.
:whatudo:
... and one has a magic MAC address and takes over as the master...
Quote from: SimonV on February 08, 2017, 04:39:38 AM
TIL that Sonos audio players participate in STP and build wireless links between devices :eek:
I just ran into a similar situation this week. A contractor was installing a Crestron audio control system, and after they tripped bpduguard on multiple ports, we discovered the panel they are installing has an "embedded switch" that is running RSTP right out of the box. Check out this quote from the manual:
QuotePrior to 2012, every digital media (DM) card and endpoint in an installation required its own IP address on the corporate network. In 2012, Crestron introduced private network mode (PNM) to DM switchers. PNM greatly reduces the number of IP addresses required for DM installations. Crestron recommends using PNM to manage Ethernet settings for DM cards and endpoints connected to a DM switcher. Other methods are not recommended.
Nice...
http://www.crestron.com/downloads/pdf/product_misc/dg_ip-considerations-guide-it-professional.pdf
Cool. IPvCrestron.
My TIL for yesterday was that layer 2 fallback on a TippingPoint IPS isn't always a layer 2 fallback. It's supposed to be a state in which the IPS doesn't do anything to the traffic, but yesterday I saw it one in fallback that was still affecting traffic.
My IPS in layer 2 fallback was doing this: :umad:
Putting the entire segment into bypass mode instead was what worked to have the IPS not affect the traffic.
So, if you have an IPS, and you tested it in layer 2 fallback... test it again.
TIL about ssh -v -v -v -v
Pretty cool stuff, ran it on the box that was closing connection and on one that wasn't to do a compare. The first box seems to not be accepting the public key of the management box, and it's happening right after an upgrade we did yesterday... hmmm...
"ssh -v -v -v -v" can be shortened to "ssh -vvvv" Most commands will accept multiple switches after a single - unless they need a value supplied. As an example
"tcpdump -n -n -i eth0" can be shortened to "tcpdump -nni eth0"
If you need more than one switch to accept a value you can still consolidate except for the ones that need values.
"tcpdump -n -n -i eth0 -w capture.pcap" can be shortened to "tcpdump -nni eth0 -w capture.pcap"
-Otanx
Quote from: Otanx on July 10, 2017, 03:09:03 PM
"ssh -v -v -v -v" can be shortened to "ssh -vvvv" Most commands will accept multiple switches after a single - unless they need a value supplied. As an example
"tcpdump -n -n -i eth0" can be shortened to "tcpdump -nni eth0"
If you need more than one switch to accept a value you can still consolidate except for the ones that need values.
"tcpdump -n -n -i eth0 -w capture.pcap" can be shortened to "tcpdump -nni eth0 -w capture.pcap"
-Otanx
TIL one more thing! Thanks, Otanx!
:tmyk:
[/me bumps Otanx' helpful rep]
TIL that the "Open" in OSPF is not a verb, but an adjective, as in "open source".
http://www.tcpipguide.com/free/t_OSPFOverviewHistoryStandardsandVersions.htm
Also that the RFC for OSPFv2 is 240 pages long. Wow. Way to RFC, dudes!
TIL how to configure SSO on F5 APM for external users
TIL that 6500 VSS is a bigger mess than I thought. Do you want to upgrade firmware? We have this cool ISSU/eFSU thing that will let you upgrade one chassis/sup card at a time. Oh, you want to upgrade from 15.1 to 15.2? You can't use ISSU. Need to reboot both at the same time.
Tomorrow I Learn (TIL) how to break a VSS with minimal down time.
-Otanx
TIL that my air conditioner won't be able to be fixed until tomorrow.
I am now one night closer to getting Hilton Diamond level this year...
Quote from: Otanx on July 21, 2017, 01:41:36 PM
TIL that 6500 VSS is a bigger mess than I thought. Do you want to upgrade firmware? We have this cool ISSU/eFSU thing that will let you upgrade one chassis/sup card at a time. Oh, you want to upgrade from 15.1 to 15.2? You can't use ISSU. Need to reboot both at the same time.
Tomorrow I Learn (TIL) how to break a VSS with minimal down time.
-Otanx
This is why I tell people go avoid VSS / stacks for anything that needs five nines uptime. Its all gravy until you need to upgrade, and/or when an upgrade or bug takes out the shared blast radius I mean control plane.
A colleague of mine watched foobar a VSS ISSU following step by step TAC instructions (TAC was on the line the whole time...) - took a hospital offline for 3 hours whilst they restored everything manually, the ISSU process wiped both sups on both chassis LOL. Mind you this was with first gen buggy VSS code, but the point stands. He has since then point blank refused to execute any VSS ISSU, always insists he needs a full reload and the short outage associated.
Quote from: Otanx on July 21, 2017, 01:41:36 PM
Tomorrow I Learn (TIL) how to break a VSS with minimal down time.
-Otanx
:lol:, good luck with that one.... :rage:
TIL (well yesterday) about Cisco FXOS
TIL about the TCP Finite State Model.
well not today, but a few days ago I learned that if you use secureCRT... back up the configs directory... life sucks when you lose all your saved sites, button bars, custom configurations, etc...
TIL that working for a company on the day of its IPO entails rather a lot of official celebrating.
Me looking at my schedule for today:
:greatoffer:
Quote from: deanwebb on October 27, 2017, 08:34:28 AM
TIL that working for a company on the day of its IPO entails rather a lot of official celebrating.
Is that why IPO stocks always seem to go down after the IPO?
Quote from: ristau5741 on November 03, 2017, 01:41:03 PM
Quote from: deanwebb on October 27, 2017, 08:34:28 AM
TIL that working for a company on the day of its IPO entails rather a lot of official celebrating.
Is that why IPO stocks always seem to go down after the IPO?
No, not really... most of that has to do with general trends in investing, initial backers taking profits up front, stuff like that.
TIL that I still had an account here.
Quote from: packetferret on November 20, 2017, 04:15:42 PM
TIL that I still had an account here.
I lol'd. :lol:
And you still remember the password. Either that, or your browser/password manager remembered it.
TIL that I'm going to learn more about Palo Alto integration with ForeScout CounterACT.
Also ServiceNow integration...
Also also SWIFT regulations...
Quote from: deanwebb on November 28, 2017, 08:00:41 AM
TIL that I'm going to learn more about Palo Alto integration with ForeScout CounterACT.
Also ServiceNow integration...
Do write up a summary on those, We've got all three products. That'd be interesting.
Yeah, that's what I have to do, eventually... write the best practices doc for CounterACT's integration with those two products.
Not today, but last friday.
I learned that the capwap tunneling on the 2800 series APs requires that you adjust the MTU (1300) because it is running click-os. Troubleshooting pretty much all day why web browsing worked in certain parts of the building and not in others when I could ping/trace-route. Kicking myself for not finding it when it screamed fragmentation.
My next step was to wireshark too...
TIL the difference between TLS RSA, TLS DHE and TLS ECDHE (and the anonymous variants thereof), followed by TLS 1.2 vs TLS 1.3 and why SSL decryption is doomed (as the server sends the cert back already encrypted with the pre-selected key share). I already knew re: certificate pinning, this seems like another nail in the coffin.
Your standard network nerd weekend link bingeing, and I don't even like studying crypto LOLOLOLOL
Quote from: wintermute000 on December 10, 2017, 02:08:08 AM
TIL the difference between TLS RSA, TLS DHE and TLS ECDHE (and the anonymous variants thereof), followed by TLS 1.2 vs TLS 1.3 and why SSL decryption is doomed (as the server sends the cert back already encrypted with the pre-selected key share). I already knew re: certificate pinning, this seems like another nail in the coffin.
Your standard network nerd weekend link bingeing, and I don't even like studying crypto LOLOLOLOL
And if we can't do SSL decryption, then metadata about web sessions will be even more important as a security metric.
Today I learned all about cipher suites, and that the Cisco WSA has a special config section where you can manually define supported and unsupported ciphers.
I also learned that not having the same config on all of your proxies can lead to some seriously bizarre behaviour.
TIL that a Dell T320 server doesn't take a GeForce 1030 card, but it will take a GeForce 730 just fine.
Also TIL that Windows Server 2016 runs Steam and Steam games just fine, especially if the underlying hardware is running a proper video card instead of the Windows Default Display Driver for the embedded onboard controller.
Also also TIL that a good video card coupled with 16 CPUs and 64GB RAM makes for a helluva gaming rig. :smug:
TBH you're probably worse off than with a higher clocked 4/6C. Them xeons don't hit the same heights as a 7700k or 8700k and for gaming workloads, a smaller number of very high freq threads is the go
RAM makes practically no difference once you have 'enough' (which is usually 16Gb)
What are you playing?
Quote from: wintermute000 on February 20, 2018, 02:17:55 AM
TBH you're probably worse off than with a higher clocked 4/6C. Them xeons don't hit the same heights as a 7700k or 8700k and for gaming workloads, a smaller number of very high freq threads is the go
RAM makes practically no difference once you have 'enough' (which is usually 16Gb)
What are you playing?
16 cores is better than anything my kids have on their student-grade laptops. :lol:
I'm playing Cities:Skylines, EU4, HOI4, Ticket to Ride... not very FPS-intensive games. I need the RAM more than anything for C:S. 64GB does the trick like a dream.
Ticket to Ride is great, only have the boardgame though :)
Quote from: SimonV on February 21, 2018, 03:57:06 AM
Ticket to Ride is great, only have the boardgame though :)
Having it on Steam means I can play it on a plane or in a hotel, which is not normally practical with the boardgame.
TIL that my product works best when it's set up properly.
Did one teensy mistake in the initial setup that totally screwed me over, took me 3 hours to finally figure it out by going back through settings I thought were entered correctly. Glad this wasn't on the certification practical.
Quote from: deanwebb on February 22, 2018, 02:55:31 PM
TIL that my product works best when it's set up properly.
Did one teensy mistake in the initial setup that totally screwed me over, took me 3 hours to finally figure it out by going back through settings I thought were entered correctly. Glad this wasn't on the certification practical.
Sometimes, it just takes a single bit to screw up everything.
TIL some Avaya switch commands. Not all that hard, once you know the basics and what you hope to accomplish.
TIL that the actual mechanism AWS uses to get YOUR END to fail over to a secondary tunnel (on the same overall VPN connection... GAH their terminology drives me nuts) is MED. Which explains why no explicit config is required on customer gateway side.
Would be nice for you guys to simply write it in the document instead of just hand waving it away ("automatically fail over").....
TIL that the snmp-server host command works with an explicit IP address and won't send to a subnet or ACL.
TIL that an XML tag that ENDS with a / like <foo/> is self closing i.e. simply shorthand for <foo> </foo>........ DOH
Quote from: wintermute000 on April 16, 2018, 05:08:40 AM
TIL that an XML tag that ENDS with a / like <foo/> is self closing i.e. simply shorthand for <foo> </foo>........ DOH
This is why well-formed HTML image code has the / at the end.
Welcome to 1998, winter! :)
:tmyk:
Today I learned about DNS glue records. apparently there were some circular references in the zone files I needed to fix.
Quote from: ristau5741 on May 02, 2018, 11:19:37 AM
Today I learned about DNS glue records. apparently there were some circular references in the zone files I needed to fix.
Glue records?
:zomgwtfbbq:
What are glue records?
Quote from: deanwebb on May 02, 2018, 11:26:03 AM
Quote from: ristau5741 on May 02, 2018, 11:19:37 AM
Today I learned about DNS glue records. apparently there were some circular references in the zone files I needed to fix.
Glue records?
:zomgwtfbbq:
What are glue records?
DNS records that glue stuff together.
A glue record is simply the association of a hostname (nameserver, or DNS ) with an IP address at the registry.
https://wiki.gandi.net/en/glossary/glue-record
Thanks Gandi
p.s. I liked this statement from that link above
"Glue records are needed when you want to set a domain's nameservers to a hostname that is a subdomain of the domain itself. "
:twitch:
In plain language, if the DNS servers your domain points to are in the same domain (hence resulting in circular logic), then you need a glue record to show what the IP is.
This month I have learned how useful mnemonics are. The "Real Women Date Engineers In Combat Armor" helped a lot with CCNA CyberOps.
Two weeks ago, I learned a ton of stuff about ServiceNow.
Chances are, I'm gonna learn a lot more about it in the days and weeks to come...
Yesterday I Learned... the part I was missing with MPLS and MP-BGP. I didn't realize that a label was included as part of the NLRI in the BGP advertisement. Once I realized that then everything clicked, and makes sense.
-Otanx
LOL wait till you get deeper in the MPLS rabbit hole like BGP-LU and unified MPLS. I LOVE this stuff but as I work in enterprise space primarily, I don't get to do this for realz (and if I wanted to I'd likely have to go back down a seniority/pay grade or two.... sigh)
TIL...
... about SNMP inform messages.
They're like traps, but will keep sending until the other side responds affirmatively that it has received the SNMP inform.
https://kb.juniper.net/InfoCenter/index?page=content&id=KB4380&cat=SNMP&actp=LIST
TIL that there's such a thing as bypass mode for UPS devices.
TIL.. people CAN get fired for incompetence in this organization. Well, removed from the work site anyway, permanent gov'ment employees are almost impossible to fire.
Nice as a person. But jeebus i'm glad we are getting a new boss that has experience :twitch:
Quote from: deanwebb on January 16, 2019, 07:49:00 AM
TIL that there's such a thing as bypass mode for UPS devices.
LOL.. I learned that lesson a while back in my network technician days. Did you dump power from the whole stack?
No, this was when a customer was trying to test our gear for HA in a data center...
TIL where to find username and domain info in Windows registry.
Computer\HKEY_CURRENT_USER\Volatile Environment
It was yesterday, but continuing today...
Wrote my first Ansible playbook. I have been using Ansible for awhile, but always using pre-made playbooks, and just editing the vars and Jinja templates. This time I couldn't reuse an existing playbook. I have a bunch of files that contain IP addresses that we need to import into our blackhole router. Each file is named the change ticket number that approved the blackhole of the IPs. I need to read in all these files into one dict. Then use them in a template to create the static routes.
I think this is really cool (and kind of funny):
My template has this line:
ip route {{ route }}/{{ ansible_bl_routes[ticket][route]["mask"]|default(ansible_bl_routes[ticket]["mask"])|default(ansilbe_bl_default_ipv4_mask) }} {{ ansible_bl_routes[ticket][route]["next_hop"]|default(ansible_bl_routes[ticket]["next_hop"]|default(ansible_bl_default_ipv4_next_hop }}
and produces the line:
ip route 1.1.1.1/32 192.0.2.1
of course it will produce that line 1,000 times with different IPs, masks, and next hops. I have a default mask and next hop that can be over ridden by a per ticket value, or a per route value. So my global defaults are /32 and null0. However, I can override the mask to say blackhole 10.0.0.0/8 on my public edge routers. By adding the following to the file;
12345:
"10.0.0.0":
mask: "8"
-Otanx
Learning all the things. In the last week I have expanded on my Ansible skill set. I have learned;
1. How to setup a python virtual environment (virtualenv /var/venv/network-ansible)
2. How to add 3rd party modules into Ansible playbooks. If they are written correctly just dump them in the right directory, and the filename is the name of the task you use in the playbook.
3. How to move a virtual environment. Just copy the entire directory structure. Then look for broken sym-links and fix them. Also may need to fix permissions on everything.
4. How to write a RPM spec file. Not 100% sure I have this figured out. It isn't failing out yet... Has been running for about 15 minutes. Maybe I acidently RPMed the entire linux install? Need to be able to package the venv to be deployed to a bunch of systems.
At home I have also installed Ansible on a Raspberry Pi, and got it to talk to my home gear. Not doing much with it yet, but it is working. Took about 45 minutes, and a lot of that was syntax errors in the playbook. This automation thing is pretty useful, and cool.
-Otanx
TIL how to submit a JSON query to JAMF.
TIL about initial setup and configuration of EtherSwitch Modules on the ISR. It sorta blew my mind that the module has its own flash memory, IOS and configuration separate from the router.
:shock:
I also learned how to do this today..
RTR#
***
***
*** Message from tty3 to all terminals:
***
I AM IN UR RTRZ
RTR#
Quote from: config t on November 04, 2019, 01:18:11 AM
I also learned how to do this today..
RTR#
***
***
*** Message from tty3 to all terminals:
***
I AM IN UR RTRZ
RTR#
:rofl:
OK, so how do you do that?
"send line" and then specify a vty line on which to send. I use it to let my colleagues know I'm keeping an eye on them, when they're doing changes :)
Quote from: SimonV on November 11, 2019, 02:56:37 PM
I use it to let my colleagues know I'm keeping an eye on them, when they're doing changes :)
stop micromanaging already >:D
Quote from: ristau5741 on November 12, 2019, 07:20:22 AM
Quote from: SimonV on November 11, 2019, 02:56:37 PM
I use it to let my colleagues know I'm keeping an eye on them, when they're doing changes :)
stop micromanaging already >:D
I found out by accident that our Opengear console servers will allow multiple connections to the same console. I wanted to check something (probably command syntax), and connected to a development router that our junior guy was working on. I left it open, and watched him configure some stuff.
-Otanx
send * will send to all open terminal sessions.
It's definitely good for the LOLs :XD:
Just make sure you don't need a change request to do that... :rofl:
send * "all your base are belong to us"
:mrgreen:
OK, so my co-workers are going to hate you all soon. So the send command gives you a area to enter a multi line entry. When it shows up on their side it has a giant Message from prompt. Howver, if I put in enough lines of message that part will scroll off the screen, and they will not see it. So send the message.
hostname#
hostname#
*repeat this for 80 lines to scroll the Message from lines off the top of the screen*
hostname#wr erase
Erasing the nvram filesystem will remove all configuration files! Continue? [confirm]
Erase complete *not sure what the real output would be, but neither would my coworkers*
hostname#reload
System configuration has been modified. Save? [yes/no]: no
Reload command is being issued on Active unit, this will reload the whole stack
Proceed with reload? [confirm]
hostname#
I am not evil. I like to think of myself as chaotic neutral.
-Otanx
Quote from: Otanx on November 14, 2019, 11:31:18 AM
OK, so my co-workers are going to hate you all soon. So the send command gives you a area to enter a multi line entry. When it shows up on their side it has a giant Message from prompt. Howver, if I put in enough lines of message that part will scroll off the screen, and they will not see it. So send the message.
hostname#
hostname#
*repeat this for 80 lines to scroll the Message from lines off the top of the screen*
hostname#wr erase
Erasing the nvram filesystem will remove all configuration files! Continue? [confirm]
Erase complete *not sure what the real output would be, but neither would my coworkers*
hostname#reload
System configuration has been modified. Save? [yes/no]: no
Reload command is being issued on Active unit, this will reload the whole stack
Proceed with reload? [confirm]
hostname#
I am not evil. I like to think of myself as chaotic neutral.
-Otanx
Just remember, you are not allowed to do anything that, when you think about, makes you laugh for more than 15 seconds!
:haha3:
Quote from: Otanx on November 14, 2019, 11:31:18 AM
OK, so my co-workers are going to hate you all soon. So the send command gives you a area to enter a multi line entry. When it shows up on their side it has a giant Message from prompt. Howver, if I put in enough lines of message that part will scroll off the screen, and they will not see it. So send the message.
hostname#
hostname#
*repeat this for 80 lines to scroll the Message from lines off the top of the screen*
hostname#wr erase
Erasing the nvram filesystem will remove all configuration files! Continue? [confirm]
Erase complete *not sure what the real output would be, but neither would my coworkers*
hostname#reload
System configuration has been modified. Save? [yes/no]: no
Reload command is being issued on Active unit, this will reload the whole stack
Proceed with reload? [confirm]
hostname#
I am not evil. I like to think of myself as chaotic neutral.
-Otanx
I think you mean MY co-workers are about to hate YOU soon >:D
:whatudo:
This would be a great prank on Candid Network Camera...
"show startup-config" is a thing
TIL more cool stuff about CrowdStrike. When we have it tapped into the Forescout, properties discovered in CS can be fed to FSCT for additional restrictions on the endpoint beyond what a CS agent may be applying.
TIL....
1) it's not easy to upgrade openstack
2) migration from one openstack to another is [...] <- fill in the blank :-[
3) I have to re-learn openstack using another deployment tool so that I can maintain and upgrade it going forward
TIL how to really ruin someone's day by disabling their AD account with a system that the AD team isn't fully aware of...
Quote from: deanwebb on January 29, 2020, 09:12:43 AM
TIL how to really ruin someone's day by disabling their AD account with a system that the AD team isn't fully aware of...
Why do I feel that went something like...
https://www.youtube.com/watch?v=DdmQIT6HyBA
-Otanx
We can also do policy loops that result in rapidly cycling on/off conditions. :smug:
TIL i learned how not to spend 24 hours on a TS call for a bad SFP, I learn from other mistakes, poor suckers... IDKWTF???
At least I got called into the TS session at 7AM, by noon I gave up and hung up the phone, exasperated. Not a firewall issue.
I also learned that my team lead was remove from contract today. I don't know it both are related.. hush hush, no discussion, no details.
but he was the most senior network security with over 6 years on contract and network. sucks to lose network knowledge like that.
Quote from: ristau5741 on January 29, 2020, 03:01:03 PM
sucks to lose network knowledge like that.
Yes it does. We went through that recently.
Also, why did it take so long to isolate it to the SFP? Did they entirely skip layer 1 and circle back at the end, or was it some kind of super wonky behavior going on? I have rearely seen those things rarely fail but when they do they either simply don't work (link down) or the log/interface outputs will point at it.
Quote from: config t on January 29, 2020, 09:36:40 PM
Quote from: ristau5741 on January 29, 2020, 03:01:03 PM
sucks to lose network knowledge like that.
Yes it does. We went through that recently.
Also, why did it take so long to isolate it to the SFP? Did they entirely skip layer 1 and circle back at the end, or was it some kind of super wonky behavior going on? I have rearely seen those things rarely fail but when they do they either simply don't work (link down) or the log/interface outputs will point at it.
it was a firewall issue, no then it was a routing issue, no, when they do file transfers, the get like 300K transfer rate. loose like 80% of ping. I guess someone doesn't know how to troubleshoot, top down, bottom up, or divide and conquor. i mean it was working, just not very efficiently.
Breaks the same place, every time: software.
Breaks in odd ways at odd times, but breaks hard: software interaction.
Isn't broken, but getting worse, until it doesn't even start at all: hardware.
TIL the difference between a filtered port and a closed port in an NMAP result.
Now, when the NMAP target is 127.0.0.1 and the result is "filtered", that's pretty interesting...
TIL what Windows LTSB is.
TIL that the thread started under a different username is still going strong!
Also, the old forums networking-forum.com has completely shut down and is not a landing page for Bluehost. Serves them right.
They really messed up after Steve sold the site. A real shame esp as a ton of regulars have gone to vendors (including Steve), it would have been a great neutral meeting ground for off the record chats.
Still a few of us old timers hanging around
BTW why did you change your username?
Quote from: packetferret on February 16, 2020, 02:50:13 PM
TIL that the thread started under a different username is still going strong!
Welcome back after a long absence! Would you like your post count associated with this account? :smug:
Kerberos!
(https://i.imgflip.com/2i8gxo.jpg)
OK, not really... just that it's something the AD guys set up and the rest of us turn on when the AD guys say it's ready. :smug:
https://fy.blackhats.net.au/blog/html/2017/05/23/kerberos_why_the_world_moved_on.html (https://fy.blackhats.net.au/blog/html/2017/05/23/kerberos_why_the_world_moved_on.html)
The sooner it dies the better, who the heck understands kerberos
Charlie knows Kerberos. He learned it from Pepe and Carol in HR. :smug:
TIL auto-summarizing EIGRP can cause Null0 routes to turn up in the routing table for entire subnets.
Quote from: config t on March 31, 2020, 08:15:37 AM
TIL auto-summarizing EIGRP can cause Null0 routes to turn up in the routing table for entire subnets.
I bet that led to laughter and jolliment amongst your colleagues and superiors. :D
Quote from: deanwebb on March 31, 2020, 11:00:00 AM
Quote from: config t on March 31, 2020, 08:15:37 AM
TIL auto-summarizing EIGRP can cause Null0 routes to turn up in the routing table for entire subnets.
I bet that led to laughter and jolliment amongst your colleagues and superiors. :D
Unfortunately my colleagues had been troubleshooting that for days and I simply looked at it and spotted the problem. In retrospect, declaring "Routing 101 is in Session!" wasn't so appropriate.
Quote from: config t on March 31, 2020, 01:02:31 PM
Quote from: deanwebb on March 31, 2020, 11:00:00 AM
Quote from: config t on March 31, 2020, 08:15:37 AM
TIL auto-summarizing EIGRP can cause Null0 routes to turn up in the routing table for entire subnets.
I bet that led to laughter and jolliment amongst your colleagues and superiors. :D
Unfortunately my colleagues had been troubleshooting that for days and I simply looked at it and spotted the problem. In retrospect, declaring "Routing 101 is in Session!" wasn't so appropriate.
(https://www.komunumo.net/forumo/Smileys/default/FatherStack.jpg)
:meeseeks:
Meanwhile one of the junior guys says to me, "Man you are so smart." And I confidently reply, "Please don't mistake my experience for intelligence." 8)
Stuck in ACTIVE is a goo thing? nO? :greatoffer:
Quote from: config t on March 31, 2020, 01:31:12 PM
"Please don't mistake my experience for intelligence."
I am stealing that. Maybe even adding to my email signature line.
-Otanx
It was last week, but need to brag.
TIL how to get PXE and Arista ZTP to both work in the same subnet. On Linux you can set the following in dhcpd.conf
if option vendor-class-identifier ~= "^Arista" {
option tftp-server-name "IP_Address_of_tftp_server";
option bootfile-name "arista-config.txt";
} else {
option tftp-server-name "IP_Address_of_PXE_server";
option bootfile-name "pxe_file_name";
}
When an Arista switch sends the Discover DHCP message it includes a vendor field that identifies it as an Arista, then the model, and serial number. What we are doing above is matching that field if it starts with Arista. If it matches set the tftp server IP, and the text file containing the Arista config we want to push that is on the tftp server. If it does not match send the normal PXE settings so the server guys stuff will keep working.
The Arista will download that file. If that file is an Arista config it saves it to startup-config, and reboots. You can also have it download a script to run instead, but I have not played with that.
-Otanx
TIL our VTCs reach out to an external VCS before they can call point to point in the same subnet. I still need to learn exactly how the traffic flow works but adding the network to our inbound/outbound VTC rules fixed the problem.
TIL that not everyone in the world can add or subtract with ease.
Quote from: deanwebb on April 08, 2020, 01:18:18 PM
TIL that not everyone in the world can add or subtract with ease.
This is probably a little harsh, but someone once said to a friend of mine, "Since this isn't working out, the world needs ditch diggers."
TIL 2 hours maintenance windows at the customer request is not sufficient for any action
tried one this morning, and we ran late, extended an hour, got customer approval on success, and spent the day troubleshooting during production outage, and eventually rolled back. this should have been a 12 hours late nigh maintenance window. there was not enough time for troubleshoot and rollback even with the hour extension.
TIL that there is an MTU setting on the client side. Now that I know it, it seems like a no-brainer, but I never really had to think about it before.
Just remember that Windows will ignore the MTU *always* when sending a cert in an EAP-TLS response.
ALWAYS.
TIL (or re-learned, not sure) the power of..
show run | exclude
I have a little project going on to generate config templates integrating an updated MBL and outbound ACL on my edge routers. The Null0 routes number in the several thousand range. Picking out the handful of legit ip routes felt impossible until I had that little epihpany.
TIL how to fix "water hammer".
:smug:
Quote from: deanwebb on May 05, 2020, 10:04:01 AM
TIL how to fix "water hammer".
:smug:
I had to deal with that a few years ago myself. I was lucky and my brother is a jack of all trades, and was able to take care of it for me.
TIL... Tripwire Enterprise supports IOS 12.4 and PIX firewalls. Nothing newer according to the documents updated in Mar2020.
-Otanx
TIL that I did a good job with a customer last week. :D
Quote from: deanwebb on April 14, 2020, 01:16:33 PM
Just remember that Windows will ignore the MTU *always* when sending a cert in an EAP-TLS response.
ALWAYS.
WHAT are you serious, it just somehow magically ignores the NIC setting?or rather it ignores PMTUD responses for some reason?
Quote from: wintermute000 on May 06, 2020, 07:19:34 AM
Quote from: deanwebb on April 14, 2020, 01:16:33 PM
Just remember that Windows will ignore the MTU *always* when sending a cert in an EAP-TLS response.
ALWAYS.
WHAT are you serious, it just somehow magically ignores the NIC setting?or rather it ignores PMTUD responses for some reason?
I think it's a NIC setting override. Because that packet with the cert can be massive, and Windows don't care.
TIL how to paste column data as a row in Excel.
Quote from: deanwebb on May 07, 2020, 01:39:38 PM
TIL how to paste column data as a row in Excel.
that's handy, I learned that a while ago.
Pivot tables are also lots of fun
TIL that even if a customer has had an architecture diagram in hand for months, that customer can still be surprised by information on it and think that I was trying to sneak something past them. And TI also L that keeping a full email archive is a powerful shield for the firey darts of a customer "surprised".
TIL that setting the DF bit on an ICMP packet will help identify max MTU size along a path.
C:\Users\config.t>ping 192.168.1.1 -l 1448 -f
Pinging 192.168.1.1 with 1448 bytes of data:
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Ping statistics for 192.168.1.1:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
C:\Users\david.stern>
I really need to deep dive TCP/IP.
Quote from: config t on May 11, 2020, 11:15:04 PM
TIL that setting the DF bit on an ICMP packet will help identify max MTU size along a path.
C:\Users\config.t>ping 192.168.1.1 -l 1448 -f
Pinging 192.168.1.1 with 1448 bytes of data:
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Ping statistics for 192.168.1.1:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
C:\Users\david.stern>
I really need to deep dive TCP/IP.
The TCP/IP Guide: http://www.tcpipguide.com/
I LOVE THAT SITE. It walks through all the RFCs, great stuff.
That is why you should not block ICMP. Yes there are ICMP types that should be blocked, but if you block them all you end up breaking things.
-Otanx
Quote from: deanwebb on May 12, 2020, 09:46:58 AM
Quote from: config t on May 11, 2020, 11:15:04 PM
TIL that setting the DF bit on an ICMP packet will help identify max MTU size along a path.
C:\Users\config.t>ping 192.168.1.1 -l 1448 -f
Pinging 192.168.1.1 with 1448 bytes of data:
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Ping statistics for 192.168.1.1:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
C:\Users\david.stern>
I really need to deep dive TCP/IP.
The TCP/IP Guide: http://www.tcpipguide.com/
I LOVE THAT SITE. It walks through all the RFCs, great stuff.
Looks pretty comprehensive. Over 1500 pages of content.
I also have "The TCP/IP Guide" which apparently I haven't opened for a while because I discovered 24 Kuwaiti Dinar under the cover that I completely forgot about (roughly $78).
That book goes with the website. I think you'll find it's the same author.
:kramer:
Well that settles it. I have to read it now.
old but this is the bible
https://www.amazon.com.au/TCP-Illustrated-Protocols-Addison-Wesley-Professional-ebook/dp/B00666M52S
TIL that airlines will only do social distancing on flights with open seats. Am I ready to get on that plane again?
:no:
After seeing *that* news story, I made up my mind that I'd rather spend 2 days driving to the client site than 1 day flying.
TIL that 1:64 scale racing is great fun, me and the missus might just get into it...
https://www.youtube.com/watch?v=wxIyzEZ9vb8
:smug: Oh yeah!
TIL how to create and deploy resources in Azure.
While doing so I had the idea of deploying a virtual machine where I can run GNS3 so I don't have to keep investing in hardware. Will have to further research exactly how much that would cost, do a comparison, and figure out if I could even import my IOS stockpile into it.
TIL that you can't duplicate hostnames in DHCP reservations in Linux. I am adding DHCP to a subnet that has always just had statics. I grabbed the ARP table, and did DNS lookups for all the IPs. If there was not DNS I marked it as "unknown". Added these all as reservations to dhcpd.conf, and it wouldn't start. Now all the hosts are unknown_ip.address.
-Otanx
TIL.. "91.3% of malware uses DNS yet 68% of organizations don't monitor it." According to Cisco anyway.
TIL that Meraki opened up more of its API. :smug:
Quote from: config t on July 01, 2020, 04:30:10 AM
TIL.. "91.3% of malware uses DNS yet 68% of organizations don't monitor it." According to Cisco anyway.
and a lot of that DNS is easy to spot if you just look. It isn't as easy as bad-malware.com, but it usually isn't that hard either.
-Otanx
Quote from: deanwebb on July 01, 2020, 10:49:18 AM
TIL that Meraki opened up more of its API. :smug:
Yeah there's PowerShell now too
TIL how to use a freeware utility to spoof my MAC address on Windows. And then that Cisco switches just delete the inactive duplicate MAC address from their tables so as not to create a conflict.
TIL single mode fiber is really cheap now. Might finally be able to make the case to do our new cable plant 100% SM. Get rid of the OM3/4 stuff. The server guys can reuse all my old optics and patch cables for their servers to top of rack.
-Otanx
TIL that services work much better when they are started.
:doh:
wired autoconfig? lol
TIL single mode optics are expensive. 11K for a 100G-LR optic, 4K for a 100G-bidi optic. Might not get my single mode cable plant. The OM4 is more expensive by about 30K, but the difference in optics is over 100K. Still going to propose it. Try to use the never replace the cable again angle. Also trying to get over my dislike for AOC instead of doing fiber at all. I will still need the fiber for other stuff, but my 100G might not need it.
-Otanx
yeah. I recall a customer freaking out about how 4x 40Gb LR optics were the same cost as a couple of switches. That vendor rubber stamp at the end of the finisar conveyor belt must be made of unobtanium tears.
My rule of thumb when guesstimating is to 5x the cost (using the big gorilla vendor starting with C as the reference, however, most are not much better)
Its a nice dream to go entirely single mode, however, is it worth the price of the optics, and when in reality everything within the same hall is fine with multi-mode.
Looks like I'm about to learn about the differences between single-mode and multi-mode fiber... carry on, gents.
Quote from: deanwebb on July 24, 2020, 09:16:06 AM
Looks like I'm about to learn about the differences between single-mode and multi-mode fiber... carry on, gents.
Got you:
Fiber 101: https://www.reddit.com/r/networking/comments/3gx5dz/ysk_if_you_dont_about_fiber_optics_and_how_they/
The advanced class: https://archive.nanog.org/sites/default/files/2_Steenbergen_Tutorial_New_And_v2.pdf
These are both really good. I think anyone that deals with fiber, or high speed networking should read them. Maybe not understand everything, especially the advanced class, but have at least seen it.
-Otanx
TIL I don't know as much about FO as I thought I did :)
Had to google the OM standards. We don't do no high speed netwrokin' around here. 1Gb backbone, baby :smug:
SFP -- Small form factor pluggable. This is the most common one you'll see these days in gigabit. Has an LC connector on it. Some people call these "mini-GBICs". Those people are idiots.
^ I lol'd. :lol:
Today I wrestled with the understanding of aggregate bandwidth in the context of a KG-175D TACLANE
I think 200Mb/s aggregate bandwidth means that both 100Mb interfaces (PT and CT) can operate at full capacity simultaneously. But doesn't that mean the throughput is still 100Mb? If that's the case, why does aggregate bandwidth matter?
Maybe it's 200Mbps until one of the 100 lines fails... ?
just semantics, until it comes to licensing :/
TIL, whell yesterday, that if you don't static set mac addresses your ASA multi context virtual interfaces that they will all use the mac address of the physical interface, causing big issues, also auto-assigning mac address is not safe, research shows, that will multiple firewalls in the same broadcast domain there is slight possibility that different firewalls may select the same auto-assigned mac address on the same broadcast domain, I also learned that there are reserved MAC addresses, like RFC1918 for IP's that can be used for this and VM configurations.
:mrgreen:
Friends don't let friends use ASAs in 2020!
Quote from: wintermute000 on July 31, 2020, 08:26:08 PM
Friends don't let friends use ASAs in 2020!
sipping the Cisco Kool-Ade.
yar, if you are a Cisco shop.
TIL that people are still using ASAs in 2020. :smug:
TIL Layer 2 is still layer 2 no matter if layer 2 was layer 2 before you knew what layer 2 was... LAYER 2 IS STILL LAYER 2. I'm done.. but for real.. everyone wants to talk smack about spanning-tree until it's a problem and then nobody seems to know how to troubleshoot spanning-tree or believe me when I ask if anyone checked layer 2 before escalating this "major" ticket and then I run some simple @*%^ing commands like.. i dunno
show spanning-tree
Quote from: config t on August 06, 2020, 05:11:40 PM
TIL Layer 2 is still layer 2 no matter if layer 2 was layer 2 before you knew what layer 2 was... LAYER 2 IS STILL LAYER 2. I'm done.. but for real.. everyone wants to talk smack about spanning-tree until it's a problem and then nobody seems to know how to troubleshoot spanning-tree or believe me when I ask if anyone checked layer 2 before escalating this "major" ticket and then I run some simple @*%^ing commands like.. i dunno
show spanning-tree
just be glad it ain't layer 2 1/2. >:D
for most BPDU's are a tough pill to swallow, 'specially with all them modes
i'm good now. a little bit of whiskey last night went a long way to calm me down haha
Quote from: ristau5741 on August 06, 2020, 08:51:14 PM
just be glad it ain't layer 2 1/2. >:D
i am intrigued. what is this layer 2.5 you speak of? MPLS? i never had the chance to work with it but it kinda blew my mind when i first learned about it. poppin' tags.
Quote from: config t on August 07, 2020, 08:14:26 AM
i'm good now. a little bit of whiskey last night went a long way to calm me down haha
Quote from: ristau5741 on August 06, 2020, 08:51:14 PM
just be glad it ain't layer 2 1/2. >:D
i am intrigued. what is this layer 2.5 you speak of? MPLS? i never had the chance to work with it but it kinda blew my mind when i first learned about it. poppin' tags.
yes, MPLS. it's a weird concept to shim in a tag between layer2 and layer 3. but it's like hitting the fastforward button to get your traffic where it needs to go.
TIL about an easier alternative to man pages
https://tldr.sh/
MPLS is awesome, unfortunately you don't get to deal with it much outside of SP core networks. Enterprise is tilting largely towards IP-based overlays.
A wise man once told me that MPLS silicon costs $$$$ because its expensive to handle variable length labels in the header, that's why we have a flood of cheap fixed header VXLAN chips and why VXLAN is now the de-facto standard, despite the fact that you could pretty much recreate VXLAN functionality via an existing mid 2000s technology (i.e. MPLS).
Quote from: config t on July 29, 2020, 05:57:00 AM
Today I wrestled with the understanding of aggregate bandwidth in the context of a KG-175D TACLANE
I think 200Mb/s aggregate bandwidth means that both 100Mb interfaces (PT and CT) can operate at full capacity simultaneously. But doesn't that mean the throughput is still 100Mb? If that's the case, why does aggregate bandwidth matter?
I am 99% sure the 200Mb/s is reference to the crypto engine. For a Delta that would be 100M/s encrypt and 100M decrypt. The Delta only has 100Mb/s interfaces. When you start dealing with the Flex you get to deal with licensing, and then you get to ask them questions if the entire 200 can be used in one direction, or if it is 100/100? I do not miss my days dealing with those.
-Otanx
TIL some really cool stuff that will be officially announced in a few days. :smug:
Quote from: Otanx on August 10, 2020, 08:56:31 AM
I am 99% sure the 200Mb/s is reference to the crypto engine. For a Delta that would be 100M/s encrypt and 100M decrypt. The Delta only has 100Mb/s interfaces. When you start dealing with the Flex you get to deal with licensing, and then you get to ask them questions if the entire 200 can be used in one direction, or if it is 100/100? I do not miss my days dealing with those.
-Otanx
Never heard of the 175F, interesting. What type of environment were you using it in? The datasheet mentions it excels in disadvantaged networks like SATCOM. 200mb - 2gb, nice.
That google rabbit hole also led me to the Nano. Fits in the palm of your hand and still has better throughput than the Delta.
I left the question answered as "yep, Deltas provide 100Mb throughput".
I don't mind dealing with a few TACLANES. Key word is few. Some of the Army bases I worked on in the past had hundreds and weren't even using GEM-X (now GEM-ONE).
Quote from: config t on August 11, 2020, 03:57:53 AM
Never heard of the 175F, interesting. What type of environment were you using it in? The datasheet mentions it excels in disadvantaged networks like SATCOM. 200mb - 2gb, nice.
The Flex is great for growth. Have a new site, and not sure how much throughput they need? Send a Flex with base license. If they start maxing that just upgrade the license instead of swapping the KG. I normally hate throughput licensing like that, but GD makes it work in this instance. Also they moved back to SFP ports so you can swap fiber types without having to replace the entire KG.
Quote from: config t on August 11, 2020, 03:57:53 AM
That google rabbit hole also led me to the Nano. Fits in the palm of your hand and still has better throughput than the Delta.
Check your numbers on the Nano. It has 120M throughput like the Delta has 200M. It is slower, but awesome for mobility.
Quote from: config t on August 11, 2020, 03:57:53 AM
I left the question answered as "yep, Deltas provide 100Mb throughput".
That is how I would answer. Along with if they are even close to 100M they should go to the Golf for future expansion. Then they would go with a Delta because cost. Main reason I love the Flex. Buy it at 100M, pay later when you max it. Easier to get past the budget people.
Quote from: config t on August 11, 2020, 03:57:53 AM
I don't mind dealing with a few TACLANES. Key word is few. Some of the Army bases I worked on in the past had hundreds and weren't even using GEM-X (now GEM-ONE).
GEM is almost mandatory. Especially if any of your devices are remote.
-Otanx
TIL how to simulate a break sequence signal by manipulating the baud rate of the terminal emulator, because my 5912 embedded services engine was being a buggy SOB and wouldn't take any break sequences to boot in rommon.
TI also L that putting "login local" on the line con 0 without setting a username/password means I am locked out of the router.
o man i do not miss this stuff. like uploading an IOS via xmodem (4 hours later....) because its so old you can't get IP in rommon
xmodem is cancer
i had yet another adventure today with the voyager ECK garbage (also cancer)
recall my previous rants about klasOS and voyager
while i was on the other side of the office today engaged in a scheduled outage my new guy directed my customer to delete a file from flash: because the instructions i had given them earlier to wipe the router didn't work (because it is klasOS "cisco-like" garbage)
this file they deleted happened to be the klasOS file. so it booted with $ (linux based, interesting)
last i heard they ended up swapping the hardware module for another on hand since commands for this POS isn't available online.
here is the link for this garbage product in case any of you come across it.
https://klastelecom.com/voyager-eck/
*edit* don't be seduced because it won a "red dot design award" for its "sleek product design" it's total overheating @&#^
TIL that Windows 10 Build 2004 rearranges the furniture as far as WMI is concerned.
The thudding sound you hear are all the custom WMI scripts that are crashing because of that.
Had an interesting troubleshoot yesterday. We spent around 6 hours trying to get a EIGRP adjacency up through a GRE we are tunneling through a Site to Site VPN over a couple FortiGate firewalls. To add another layer of complexity, there are TACLANEs involved and that link in turn is also being tunneled through another Site to Site VPN.
The adjacency was flapping every 1 minute 25 seconds. So that sent me down a rabbit hole of EIGRP t-shoot'n. I observed hello packets reaching both sides but the adjacency would reset due to retransmit timeouts. So, multicast was traversing fine but the EIGRP ACK packets were being received by the other side and not sending them back.
We looked at everything on the list of possible issues according to documentation and nothing was working. I got hung up on thinking it was MTU because when I tried to ping across the link using the max configured MTU size (which EIGRP uses for the ACK unicast retry), DF bit set, etc, I was getting this weird output of
!!.!!.!!.!!.!!.!!.!!.!!.!!.!!....................
It turned out EIGRP was being filtered via control plane policy on that particular router. We aren't doing it anywhere else on the network so nobody even though about if that wonky ACL we saw had anything to do with it since it wasn't applied to any interface or process. We found it completely on accident.
Felt like a rookie move but that is a lesson I will never forget.
Wow, that's complicated stuff.
TIL how to generate an API key on a Panorama system using curl.
Got to love the government networks. IPSec, inside IPSec, inside IPSec. Usable MTU? 800. Oh, and for security we are going to block ICMP everywhere so hope you don't want to use ICMP unreachables to do TTL discovery.
-Otanx
Quote from: Otanx on September 18, 2020, 04:28:34 PM
Got to love the government networks. IPSec, inside IPSec, inside IPSec. Usable MTU? 800. Oh, and for security we are going to block ICMP everywhere so hope you don't want to use ICMP unreachables to do TTL discovery.
-Otanx
a bunch of years ago I worked with a network like that, it was almost impossible to troubleshoot. couldn't ping anything, so there was no trace route. ICMP is fine, buy only allow specific code type through the use of ACL's.
This is why I got TCPing, so I can ping via a TCP packet instead of an ICMP one.
Quote from: Otanx on September 18, 2020, 04:28:34 PM
hope you don't want to use ICMP unreachables to do TTL discovery.
-Otanx
that's a pretty old school way of saying traceroute
Quote from: config t on September 23, 2020, 05:06:56 AM
Quote from: Otanx on September 18, 2020, 04:28:34 PM
hope you don't want to use ICMP unreachables to do TTL discovery.
-Otanx
that's a pretty old school way of saying traceroute
Ha, I didn't notice that. You are right that is a weird way to say traceroute. That should have been path MTU discovery.
-Otanx
I was trying to figure out if you meant traceroute or MTU discovery :XD:
Now I'm thinking of that great Tracer-T video...
:haha1:
TIL something about RIP when I ran some debugs to figure out why my tunnel hub source loopback wasn't making it to a spoke router.
During troubleshooting one of the guys tried to move a DMVPN tunnel hub to the next hop router and left the source loopback configured there, but admin shutdown.
Apparently RIP, when it sees a network advertised from another source that it also has configured locally - even if it is shutdown - will not forward that advertisement to ANY participating interface.
So for example, my router received an advertisement for 10.0.222.255 and has the following configured:
Loopback 222
ip address 10.0.222.255 255.255.255.255
shutdown
end
!
router rip
network 10.0.0.0
no auto-summary
TIL that we have a customer that is asking for a complicated workaround because they don't want to block traffic on a particular port coming in from the VPN.
It's kind of like they're asking us to help tape their legs to their heads so they don't hurt their feet walking on rough ground... but telling us that our suggestions to wear shoes are unacceptable workarounds...
Quote from: deanwebb on September 29, 2020, 01:38:56 PM
TIL that we have a customer that is asking for a complicated workaround because they don't want to block traffic on a particular port coming in from the VPN.
It's kind of like they're asking us to help tape their legs to their heads so they don't hurt their feet walking on rough ground... but telling us that our suggestions to wear shoes are unacceptable workarounds...
which port if I may ask?
80? 443? 500? 3389?
It's a port used by every single Windows device for management. :-\ They don't want it to talk with $VENDOR on the VPN because it's out of scope due to licensing concerns... running GPO scripts to disable/re-enable services based on location is going to be way more trouble than, say, getting the firewall or IPS to just block the packets and drop the sessions.
***
TIL there's a burgeoning IoMT market, M meaning "medical". I previously thought that there were only a few players in that space, but there's more there than what met my eye previously.
TIL that just because you buy a license for an integration feature on one platform, that doesn't mean you've bought the corresponding license for the other platform.
:doh:
Quote from: deanwebb on October 07, 2020, 06:02:33 PM
TIL that just because you buy a license for an integration feature on one platform, that doesn't mean you've bought the corresponding license for the other platform.
this is the most 2020 statement here
Quote from: packetferret on October 18, 2020, 03:23:04 PM
Quote from: deanwebb on October 07, 2020, 06:02:33 PM
TIL that just because you buy a license for an integration feature on one platform, that doesn't mean you've bought the corresponding license for the other platform.
this is the most 2020 statement here
:haha3:
Funny because it's true. And now I know *one more thing* to check before we try to fire it up in a working session.
Today I (re)learned to never trust the customer when they tell me they made a configuration change I directed them to do. Always ask for config outputs and screen shots of GUI configs.
Quote from: config t on October 20, 2020, 05:41:52 AM
Today I (re)learned to never trust the customer when they tell me they made a configuration change I directed them to do. Always ask for config outputs and screen shots of GUI configs.
ALWAYS
And if you can see those configs live in a screen share, so much the better! I have one customer where I trust it only if I see a screen shot from one of my co-workers or I see it myself. I don't trust them to do a screen shot to save their networks...
The worst part is that they are the customer so I can't call them out on it. I have to pretend like maybe the system isn't working right... Hey, it isn't working, can you make sure the system applied your changes? Then they use that as the excuse like oh hmm, it reverted my changes I made them again please test. I know they didn't do it. They know they didn't do it. But we all have to pretend the device did something weird.
-Otanx
Quote from: Otanx on October 20, 2020, 01:39:34 PM
I know they didn't do it. They know they didn't do it. But we all have to pretend the device did something weird.
-Otanx
:vendors:
Gotta be diplomatic, as well, so the customer doesn't open up a ticket with support to investigate why the device didn't commit changes and somehow dropped all the logging about any change activity with the changes that didn't get committed...
This is a small environment and my legend has been growing over the past year, so I get a little bit of grace to call people knuckleheads occasionally, which can be useful.
Quote from: config t on October 22, 2020, 02:13:10 AM
This is a small environment and my legend has been growing over the past year, so I get a little bit of grace to call people knuckleheads occasionally, which can be useful.
Cherish those moments.
CHERISH them.
Quote from: config t on October 22, 2020, 02:13:10 AM
This is a small environment and my legend has been growing over the past year, so I get a little bit of grace to call people knuckleheads occasionally, which can be useful.
Our internal teams I can get away with this(and I do). Over the years they have learned to troubleshoot before escalating. It is nice when I get an email and it has log messages included. Now I just need them to understand what (no connection) in an ASA deny means. They will figure it out about the time we change to another vendor.
-Otanx
Sadly a lot of my troubleshooting requests start with, "Hey can you check the network and see if anything is going on?" No context, no information, not even telling me WHICH network.
Quote from: deanwebb on October 22, 2020, 09:43:47 AM
Quote from: config t on October 22, 2020, 02:13:10 AM
This is a small environment and my legend has been growing over the past year, so I get a little bit of grace to call people knuckleheads occasionally, which can be useful.
Cherish those moments.
CHERISH them.
:XD: It will probably be somewhat bittersweet when it's time to move on in a few years.
well, today I relearned that MTU is not MSS,
With an MTU of 1500, an MSS segment size of 1460, through an IPSEC tunnel, the packets got dropped
With an MTU of 1500. as MSS segment side of 1426, through an IPSEC tunnel, the packets went through.
little things sometimes get forgotten....
Quote from: ristau5741 on November 18, 2020, 12:36:43 PM
well, today I relearned that MTU is not MSS,
With an MTU of 1500, an MSS segment size of 1460, through an IPSEC tunnel, the packets got dropped
With an MTU of 1500. as MSS segment side of 1426, through an IPSEC tunnel, the packets went through.
little things sometimes get forgotten....
This is the TIL for me, as well. That's good to know.
Quote from: ristau5741 on November 18, 2020, 12:36:43 PM
well, today I relearned that MTU is not MSS,
With an MTU of 1500, an MSS segment size of 1460, through an IPSEC tunnel, the packets got dropped
With an MTU of 1500. as MSS segment side of 1426, through an IPSEC tunnel, the packets went through.
little things sometimes get forgotten....
I still struggle a little bit to understand what MSS is and does.
Max segment size, usually set at the application layer, in our case it was Oracle SQL configuration on the server.
Strictly speaking its a TCP thing
"Essentially, the MSS is equal to MTU minus the size of a TCP header and an IP header:
MTU - (TCP header + IP header) = MSS
One of the key differences between MTU and MSS is that if a packet exceeds a device's MTU, it is broken up into smaller pieces, or "fragmented." In contrast, if a packet exceeds the MSS, it is dropped and not delivered."
TIL object-groups in IOS are a thing and now I am looking at our ACLs like :eek:
Quote from: config t on November 24, 2020, 10:35:47 PM
TIL object-groups in IOS are a thing and now I am looking at our ACLs like :eek:
If they were put in by a contractor that got paid by the line of code, you better believe there's one line for every possible source-destination-port combination.
object-groups are not an auditors friend, especially for high-risk protocols, it is hard to audit, track, and remove to keep the networks safe.
example
Fred at 1.1.1.1 wants to remote desktop to 2.2.2.2, on december 30th, this rule will expire in 180 days.
Mary at 1.1.1.1 wants to remote desktop to 2.2.2.2, on January 15th, this rule will expire in 180 days.
Now, how does one track the expiring rules in an object group?
Personally, I would use a Host to host ACL for each, and use a time range to disable after 6 months
In Cisco ASA terms, something like,
# time-range thru9-30-2021
# absolute end 00:00 01 October 2021
# periodic daily 0:00 to 23:59
# access-list outside_access extended permit tcp host 1.1.1.1 host 2.2.2.2 eq 3389 log
Today I learned about RJ.5 connectors. Kind of like USB-C for CAT-5e. Almost half the width of RJ-45. From some searching it looks like it can do 1G and 60W POE. Downside is it isn't an open standard. Looks like it is a proprietary standard by TE Communications. My brother handed me one last night. His company uses them for some internal cabling inside their devices where space is at a premium.
-Otanx
WTF!!! lol i guess you learn something new every day
TIL that our CEO makes comments on our developer's github. Since the comments supported my view in a dispute, I am all like:
:greatoffer:
I like to think they comment on random code with stuff like "GIT GUD SCRUB", or maybe a simple "interesting"
-Otanx
This comment was, "Treat it as a defect, fix it." :D
TIL how to create website whitelist rules in FortiGate after a firmware update broke something in UTM. Now I'm learning about UTM too :)
TIL that I can learn a lot from the TIL posts.
Quote from: heath on February 22, 2021, 10:57:46 AM
TIL that I can learn a lot from the TIL posts.
:awesome:
WIN
TIL where the WSUS registry key is located on Windows.
TIL that we have a GitLab instance that the systems/server team have been using extensively and that I have access to it.
TIHTL (Tomorrow I Hope To Learn) how to have Oxidized (which already uses Git to backup network configs to a local repository) push configs to GitLab for additional off site backup redundancy.
I setup Oxidized to do that a couple years ago, and it was pretty easy. If you need help let me know, and I can see if I have any of the config around somewhere.
-Otanx
TIL no matter the critical project you are working on, something will creep in with more urgency, so my task due tomorrow got upended with a task due in 2 hours.
Quote from: Otanx on March 03, 2021, 09:47:32 AM
I setup Oxidized to do that a couple years ago, and it was pretty easy. If you need help let me know, and I can see if I have any of the config around somewhere.
-Otanx
Thanks! I may have to take you up on that. My install of Oxidized is actually as a plugin for LibreNMS and I don't want to break that.
I don't know how it links with LibreNMS. Ours was the basic docker image. We had it use http as a source, and it would query Netbox for the list of hosts to backup. Then save as git, and use the "hooks" function to push that to our internal repo. For the hooks we did the following...
hooks:
push_to_git:
type: githubrepo
events: : [post_store]
remote_repo: your_git_server:the_repo_name.git
username: your_user
password: your_password
Insert your repo, username, and password. You should be able to add that to the Oxidized config file. I just looked at the Oxidized docs, and our config is just a copy and paste of the example so it apparently just works. We don't use groups. If you do the docs have an example, but you have to have different repo for each group.
-Otanx
TIL not to snmpwalk the entire MIB tree for a device with full BGP tables. Three hours later and still walking. File I am saving to is up to just over 1GB. The MIB contains multiple entries for each route in the table, and I think it also includes entries for each received route from your BGP neighbors. Maybe it will finish today. At least I will have a good file to use for my SNMP project.
-Otanx
TIL how to check a web page for accessibility.
Fun fact: there are over 100 A and AA errors on these forums.
Today I received a 13" MacBook Pro, with no instructions on how to use, I am not a Mac person. :twitch:
Quote from: ristau5741 on March 09, 2021, 04:15:39 PM
Today I received a 13" MacBook Pro, with no instructions on how to use, I am not a Mac person. :twitch:
The equivalent of the CTRL and ALT keys are reversed which really screws up keyboard shortcut muscle memory. For instance, to copy and paste, you have to think ALT-C and ALT-V instead of just letting your fingers do what they've done for decades and CTRL-C and CTRL-V.
You may notice there is a Delete key, but no Backspace key. The Delete key is the Backspace key. It's labeled Delete, but functions like Backspace. There is no Backspace key that functions like Delete.
File dialog boxes are often limited in functionality to the purpose they were opened for. For instance in a "Save File" dialog box in Windows, you can also rename, copy, move or delete other files, create folders, etc. On a Mac, you're pretty much limited to saving the file.
That's what I remember from my brief attempt a few years ago to convert to Mac and the main reasons that conversion failed.
Quote from: ristau5741 on March 09, 2021, 04:15:39 PM
Today I received a 13" MacBook Pro, with no instructions on how to use, I am not a Mac person. :twitch:
Those things make excellent heat sink risers for PC laptops!
TIL about network rebooters. Some will reboot when links go down, some will reboot a device after every (X) days of uptime.
Handy to have for certain Cisco bugs in ASA code... :smug:
TIL about " | json" on Arista. Use it for show commands and instead of getting the normal text output to read through it will output the data formatted in JSON. Very helpful if you are writing scripts that have to parse the output of show commands. Get rid off all my ugly regex. Now if only it worked on Cisco.
-Otanx
yeah arista is crack for scripters/python junkies though IIRC you can do it now in NX-OS as well (defo can do it in NX-API, duh)
Quote from: deanwebb on March 10, 2021, 02:59:11 PM
TIL about network rebooters. Some will reboot when links go down, some will reboot a device after every (X) days of uptime.
Handy to have for certain Cisco bugs in ASA code... :smug:
We have some network rebooters on our night shift. Except they just reboot the devices any time there is an issue, and then give up when it doesn't solve the problem.
Quote from: config t on March 14, 2021, 01:38:59 AM
Quote from: deanwebb on March 10, 2021, 02:59:11 PM
TIL about network rebooters. Some will reboot when links go down, some will reboot a device after every (X) days of uptime.
Handy to have for certain Cisco bugs in ASA code... :smug:
We have some network rebooters on our night shift. Except they just reboot the devices any time there is an issue, and then give up when it doesn't solve the problem.
:haha1:
TIL about port-forwarding. One of our integrators was having issues with an application talking across an internal layer 3 boundary and it turns out the app is designed to broadcast some of the traffic. Then I learned that DHCP relay (ip helper) is just an easy button for port-forwarding of DHCP/PXE broadcast traffic.
That it is indeed. And I learned a few months ago that it's possible to have too many DHCP relays on a system. Usually, customers don't find that out until they've brought up their 20th separate DHCP system, but this one did!
DHCP servers, DHCP servers everywhere. Why so many.
Quote from: config t on March 31, 2021, 01:07:57 PM
DHCP servers, DHCP servers everywhere. Why so many.
for network performance, nothing like 20 DHCP servers chatting across the network vying for assigning some IP to a client.
TIL that I set up my UPS correctly! :smug:
Today (well yesterday) I re-learned that one cannot ping an ASA inside interface from outside. :(
Really? Is that a hard rule or can you create a policy to allow ICMP from specific hosts/networks.
It is a hard rule. You can't access an ASA interface on the far side of where the packet came in. The only exception is for packets that come in on an IPSec tunnel terminated on the ASA. This also isn't just ICMP. Telnet, SSH, SNMP, etc. We have the same issue. Our health monitoring server is on a different interface than the backup server, and those are both different than our management clients. So DNS for our ASA resolves to the client interface, and we have host files on the servers that override and supply the IP of the closest interface. It is kind of stupid.
-Otanx
Wow. I can only imagine how much time I would have wasted trying to figure that out if I ran into it in the field.
Quote from: config t on April 04, 2021, 03:59:13 AM
Wow. I can only imagine how much time I would have wasted trying to figure that out if I ran into it in the field.
hours, only 2-3 in my case
LOL at a FW that can't act as a DNS proxy
anyway... here's a tricky, terrible hack-around for this: https://herdingpackets.net/2014/02/20/faking-an-asa-as-a-dns-forwarder/
Quote from: ristau5741 on April 04, 2021, 07:00:42 AM
Quote from: config t on April 04, 2021, 03:59:13 AM
Wow. I can only imagine how much time I would have wasted trying to figure that out if I ran into it in the field.
hours, only 2-3 in my case
And that was this time? :XD:
TIL:
When considering a SPAN session, "Overruns" happen when there are too many packets to deal with. "Dropped" packets are IPv6 frames when the interface is not set up for IPv6, unintended VLAN tags, and similar ignored packets.
And because that came from another person who learned it today, LEARNCEPTION!!!!
Quote from: deanwebb on April 22, 2021, 08:42:32 AM
TIL:
When considering a SPAN session, "Overruns" happen when there are too many packets to deal with. "Dropped" packets are IPv6 frames when the interface is not set up for IPv6, unintended VLAN tags, and similar ignored packets.
And because that came from another person who learned it today, LEARNCEPTION!!!!
Like with our favorite Cisco ASA product, where the interfaces listen to everything, and anything not destined for the firewall is considered "dropped" e.g. broadcasts etc.
Be careful, on some switch architectures dropping on the SPAN port = dropping on the actual port, exactly like blocking water coming out of an outlet
TIL.. (or re-learned, not sure) TACLANEs won't form a security association (SA) until traffic is being generated on the Plain Text side by networks other than the Cipher and Plain text networks. Why? Because TACLANEs. It took eight of us 3 days of scratching our heads at perfectly configured hardware to figure that out. Apes Strong Together!
Quote from: config t on May 06, 2021, 11:13:10 AM
TIL.. (or re-learned, not sure) TACLANEs won't form a security association (SA) until traffic is being generated on the Plain Text side by networks other than the Cipher and Plain text networks. Why? Because TACLANEs. It took eight of us 3 days of scratching our heads at perfectly configured hardware to figure that out. Apes Strong Together!
:developers:
TACLANEs!
The thing with TACLANEs you need to remember is they are just IPSec tunnel devices that use very special keys. You are correct they will not build an SA without interesting traffic. Also the PT interface needs to be up/up.
-Otanx
Well said. I actually paraphrased that in my issue/resolution recap.
Another thing to remember about TACLANEs is there is a ton of stuff we need to remember about TACLANEs :XD:
Yep, a good COMSEC guy is worth their weight in gold. It is a very specific skill set that is only used in government so many people don't want to deal with it. I was so happy when my last day of COMSEC came, and I got to debrief.
-Otanx
TIL
Not following SOP is considered non-compliance
non-compliance is considered insubordination
insubordination is cause for dismissal.
C:-)
(I didn't learn the hard way)
Our TACLANE issue came back again on Sunday. We must have got lucky when it formed the SA, because they tried to connect it again and it kept failing with the same error messages (IKEv1 timeout) it turned out, and as I had theorized, a recent Black Core change required us to lower the MTU on our transport and nobody told us that tidbit. We figured it out after another several hours of troubleshooting.
My guess for the SAs that are already formed with the head TACLANE is that the maintenance traffic for maintaining the SAs is less sensitive to drops and retransmits. I bet if they had dropped they wouldn't have come back up.
Quote from: Otanx on May 07, 2021, 10:19:52 AM
Yep, a good COMSEC guy is worth their weight in gold. It is a very specific skill set that is only used in government so many people don't want to deal with it. I was so happy when my last day of COMSEC came, and I got to debrief.
-Otanx
Every environment I've been in the COMSEC guy just issues key mat or might go as far as to do the fill as well. It's always been up to the network team to configure and troubleshoot.
TIL (gained) a deeper understand of traffic shaping and policing.
TIL that hard water deposits sometimes come out of the pipes in a glob... and that they will turn your showering experience into a gray, sticky, mess.
TIL that sh!t gets real really fast when an IP address from China turns up where it shouldn't.
Good news is that it looks like someone typed in another IP address incorrectly. But, still...
TIL that if you have to say "however" when discussing a candidate for hiring, you don't have a strong candidate.
Quote from: deanwebb on May 21, 2021, 09:16:13 PM
TIL that if you have to say "however" when discussing a candidate for hiring, you don't have a strong candidate.
I wish I knew what it was like to have a strong candidate...
Quote from: heath on May 26, 2021, 11:38:58 AM
Quote from: deanwebb on May 21, 2021, 09:16:13 PM
TIL that if you have to say "however" when discussing a candidate for hiring, you don't have a strong candidate.
I wish I knew what it was like to have a strong candidate...
You guys crack me up. Out of a team of 7 I have 2 contractors on the O&M side who could actually tell you what the network looks like.
TIL more OT stuff.
Its ironic isn't it, its such a goldmine but its probably the most painful vertical ever, and populated by total luddites (in IT terms). They should have never let electricians configure a network because they could get their best-buy wifi 'router' up and running.... then there's all the awful 'protocols' designed to operate on no battery and 64kb RAM and other such limitations, yech
Painful vertical is correct... and so much of it has zero consideration for security, from the line devices on up to the controlling software.
TIL that in the USA, you can't run a mainframe in your residence and expect to get away with it, should it cause radio interference: https://transition.fcc.gov/bureaus/oet/info/documents/bulletins/oet62/oet62rev.pdf
TIL when configuring VXLAN don't map every vlan to a vni. By every vlan I mean 1 - 4094 minus 1002-1005. This prevents the system from building the dynamic vlan/vni entries it needs to support L3 over VXLAN.
-Otanx
TIL that there is a cool RADIUS stress-testing tool for around US $25 https://networkradius.com/radius-performance-testing/
THE PREVIOUS LINK WAS NOT A SPONSORED LINK AND IS PROVIDED FOR ENTERTAINMENT PURPOSES ONLY PLEASE DON'T REPORT ME TO THE ADMINS OR I WILL HAVE TO BAN MYSELF!!!
Not even seeing the $25 looks like they are supplying it for free.
TIL that if you fat finger the SYN Flood protection on a Palo Alto firewall and set the activate value to 000 instead of 1000 it will activate flood protection on the first TCP SYN packet from each host on your network.
-Otanx
Quote from: Otanx on June 16, 2021, 12:37:40 PM
Not even seeing the $25 looks like they are supplying it for free.
TIL that if you fat finger the SYN Flood protection on a Palo Alto firewall and set the activate value to 000 instead of 1000 it will activate flood protection on the first TCP SYN packet from each host on your network.
-Otanx
So instead of a SYN flood, you get a SYN desert! :lol:
TIL that all single mode MTP/MPO connectors use APC terminations. Was a little worried when my new fiber all showed up with green connectors. Thought I just made an expensive mistake, and got the wrong cables. After a lot of searching I found one reference that said they are all APC. Looking at everyone selling them they all have the green connectors.
-Otanx
TIL about "Martian Source" packets. :smug:
Always chuckle when I see Martian packets term used. Not sure where you ran into it, but you should also look up BCP38 (aka RFC2827). It is a short document explaining that you should filter inbound traffic, and drop traffic with invalid IP sources.
-Otanx
I'm very thankful for all the guys in networking documentation that had a great sense of humor.
TIL about EAP Chaining. It allows machine and user authentication to occur within the same EAP/RADIUS session.
^Which is fun times, should one or the other go wrong. "But the other cert is good! Why can't I get on?"
TIL that when the buffers are full up, the packets will drop. :D