Current Mistakes Thread

deanwebb · May 19, 2015, 09:46:51 AM

Just realized I sent the wrong IP address for an A record in DNS.

Which is easier, changing the record, or switching functions between the box that *should* have had its IP address submitted and its mate?

In this company, the switch is easier.

Reggle · May 19, 2015, 12:39:43 PM

Quote from: deanwebb on May 19, 2015, 09:46:51 AMIn this company, the switch is easier.

Surprisingly enough that's the case in many companies I've seen. One would say a decent DNS setup and procedure wouldn't be expensive, nor complicated.

deanwebb · May 19, 2015, 12:45:40 PM

The person in charge of external DNS is a marketer in charge of brand identity.

We had to explain the difference between a host and a subdomain to this person. The one in *charge* of DNS.

Nerm · May 19, 2015, 03:11:22 PM

Quote from: deanwebb on May 19, 2015, 12:45:40 PM
We had to explain the difference between a host and a subdomain to this person. The one in *charge* of DNS.

I couldn't find a good facepalm emoticon so this will have to do here.

deanwebb · May 19, 2015, 03:43:20 PM

The summary of the email conversation:

Network: We would like an A record for our new host, please.

DNS/Marketing: OK, we need some more information before we create your subdomain.

Network: No, we would like a host. We do not want a subdomain.

DNS/Marketing: As soon as we get that information, we can get to work on the subdomain request. (In direct reply to the message above.)

Network: No. We want a host. Just a host. Just one A record.

DNS/Marketing: Ohhhhh, wait a minute... you want a host!

Network: Yes, that is what we want.

DNS/Marketing: So should we cancel the ticket for the subdomain creation, or are you still planning to submit the paperwork for that?

Mowery · May 19, 2015, 03:44:31 PM

Dieselboy · June 02, 2015, 09:30:25 PM

Quote from: deanwebb on May 19, 2015, 03:43:20 PM
The summary of the email conversation:

Network: We would like an A record for our new host, please.

DNS/Marketing: OK, we need some more information before we create your subdomain.

Network: No, we would like a host. We do not want a subdomain.

DNS/Marketing: As soon as we get that information, we can get to work on the subdomain request. (In direct reply to the message above.)

Network: No. We want a host. Just a host. Just one A record.

DNS/Marketing: Ohhhhh, wait a minute... you want a host!

Network: Yes, that is what we want.

DNS/Marketing: So should we cancel the ticket for the subdomain creation, or are you still planning to submit the paperwork for that?

This is basically every conversation of mine with any other company providing a service or support that I have ever dealt with. I have to hide sharp opjects so I don't stab myself in the eyes.

Mowery · June 03, 2015, 07:07:08 AM

Configured 7 MGCP Voice gateways and 8 3560s that will be used at a customer's remote sites. Forgot to add DHCP Pools for the phones to the first two routers. They are on the bottom of the stack. Easy to fix but I've annoyed myself.

SimonV · July 22, 2015, 06:35:04 AM

Not my mistake but anyway

Code Select

[edit security]
  'policies'
    Policy is out of sync between RE and PFEs: cluster1.node1, cluster1.node0. Please resync before commit.
error: configuration check-out failed

{primary:node0}[edit]

Recommended solution: http://www.juniper.net/documentation/en_US/junos12.1/topics/task/operational/security-policy-security-device-synchronizing.html

deanwebb · July 22, 2015, 09:29:53 AM

wintermute000 · July 22, 2015, 07:17:29 PM

Quote from: SimonV on July 22, 2015, 06:35:04 AM
Not my mistake but anyway

Code Select Expand
[edit security] 'policies' Policy is out of sync between RE and PFEs: cluster1.node1, cluster1.node0. Please resync before commit. error: configuration check-out failed {primary:node0}[edit]

Recommended solution: http://www.juniper.net/documentation/en_US/junos12.1/topics/task/operational/security-policy-security-device-synchronizing.html

that1guy15 · July 22, 2015, 07:44:31 PM

My last was trying to train a couple juniors how to add a FEX to a pair of 5Ks with vPC. I was lighting up a new FEX like I have done 1 milion times.

This fex was going to be 102 and I build all the config in notepad to lay out what was needed for them. When I pasted the config in I got an error on the Port-channel config. Nothing seemed wrong with the config on Port-Channel02 so I figured it was a one-off issue. I decided to remove the port channel and re-add it.

Code Select

config t
no interface port-channel02

I then manually added Po102 and showed them how to a FEX is added. All went smooth

About 5 minutes later everything in the DC blew the f* up... If you already picked up on what I did props to you!

If not you will see that port-channel02 is not PO102. Thats a f'n "L". I just dropped PO2 not Po102...
Shit, shit,shit.. Po2 is the uplink between my 5Ks and 7Ks. I just dropped my whole DC!

Took me 5 minutes to figure it out and correct but my apps team spent the next 4 hours stabilizing their shit.

Sometimes I hate this shit...

Otanx · July 22, 2015, 09:33:01 PM

Funny you post that today. I did something very similar late last week. Was setting up ports for a server migration from our old 3560s to our new FEX setup. Copied the port configs on the 3560, and modified them to the Nexus syntax. Forgot to validate the Po numbers were not in use on the 5Ks. Pasted the config and everything seemed OK. Get a call from a systems guy. One of their servers isn't responding. Took me about 10 minutes to figure out that one of the Port Channels was in use, and when I pasted in the config I moved it to a new vlan. Oops.

-Otanx

killabee · July 22, 2015, 10:19:38 PM

I was going through our ISE policies and saw a policy rule labeled "TEST" with object elements also containing the word "test." I think, "Hmmm, this pesky test rule has been here forever and I'm tired of looking at it. I'm going to disable it!"...

Several days later I get a ticket that certain users have been having wireless issues for several days. Sure enough, that "test" rule was serving a production function. It pissed me off pretty badly. I'm OK making my own mistakes, but when someone drops a banana peel for me to slip on.....that's not cool.

wintermute000 · July 22, 2015, 10:37:29 PM

Quote from: that1guy15 on July 22, 2015, 07:44:31 PM
My last was trying to train a couple juniors how to add a FEX to a pair of 5Ks with vPC. I was lighting up a new FEX like I have done 1 milion times.

This fex was going to be 102 and I build all the config in notepad to lay out what was needed for them. When I pasted the config in I got an error on the Port-channel config. Nothing seemed wrong with the config on Port-Channel02 so I figured it was a one-off issue. I decided to remove the port channel and re-add it.

Code Select Expand
config t no interface port-channel02

I then manually added Po102 and showed them how to a FEX is added. All went smooth

About 5 minutes later everything in the DC blew the f* up... If you already picked up on what I did props to you!

If not you will see that port-channel02 is not PO102. Thats a f'n "L". I just dropped PO2 not Po102...
Shit, shit,shit.. Po2 is the uplink between my 5Ks and 7Ks. I just dropped my whole DC!

Took me 5 minutes to figure it out and correct but my apps team spent the next 4 hours stabilizing their shit.

Sometimes I hate this shit...

I am not envying what you will have to go through re: PIR and other political knock on effects.
It happens to the best of us sometimes!

I recall at my last job, one of our guys threw in a debug ntp server and that crashed a 6500 VSS (IOS bug).
Thereafter, they decided that all debug commands required change management as well. sigh