It's not that they're idiots... they're just not as well-informed about networking as I am...
People that call in to say that they can't get an application to work, then blank out when I ask for the IP address of the server... but still expect me to solve the problem, and then never mind their reaction when I ask them what TCP ports are involved in the communication!
This thread is for venting. I find that it helps to deal with problems by venting about them. Then I don't have to shout at the end-users. Feel free to add your vents, as they come to you.
Oooh, I have enough material to keep this thread chugging for months.
Quote from: vito_corleone on January 04, 2015, 09:13:07 PM
Oooh, I have enough material to keep this thread chugging for months.
oh oh :excited: let's hear it :drama:
Got a call this morning, turns out we had a circuit down hard and no back up for this particular site. About an hour later I get another call from the "trusted" server guys. Their complaint was that their servers are saying that they lost connection to the corp NTP server. . . . . I wanted to throw the phone across the room. I had to explain to them (AGAIN) that since their circuit is down that their connections to HQ and the Internet will be down. :matrix:
Quote from: Ironman on January 04, 2015, 10:12:26 PM
Got a call this morning, turns out we had a circuit down hard and no back up for this particular site. About an hour later I get another call from the "trusted" server guys. Their complaint was that their servers are saying that they lost connection to the corp NTP server. . . . . I wanted to throw the phone across the room. I had to explain to them (AGAIN) that since their circuit is down that their connections to HQ and the Internet will be down. :matrix:
I've seen that dog and pony show before. Half of me wants to string them along an ask them a bunch of basic questions like, are you able to check your email? Can you get to any website on the internet? Maybe its a DNS issue, are you able to ping by IP address, try 127.0.0.1? Then the other (superior) half of me remembers that *most* techs are a bunch of derps and it's better for my health to lie, and tell them I've opened a ticket.
Quote from: mynd on January 04, 2015, 09:32:15 PM
Quote from: vito_corleone on January 04, 2015, 09:13:07 PM
Oooh, I have enough material to keep this thread chugging for months.
oh oh :excited: let's hear it :drama:
Most of mine are generic customer-related things. Like:
Why can't you have these 7Ks (which haven't shipped yet) deployed in 5 days?!
We need that obscure new feature! Why are you hesitant to deploy the newest .0 code release!?
I need to maintain 100% uptime while you replace every piece of our hardware during this massive core/DC migration!!!
I'll try to think of specific stories I can post without having to worry about someone stumbling across it at some point.
Gotta love the demands for 100% uptime, no matter what. Or 99.9999% uptime, which is even more annoying. Have an outage with 100% uptime and, oh well, won't make that goal this year... while we're at it, though...
Outage with 99.9999% uptime means you have exactly 17.4 seconds to resolve the outage before the ticket starts escalating. No fun at all.
My favorite is when a client says "Is there ANY chance that this will bring the network down" when we're troubleshooting an issue that had their device randomly crashing.
Management: "We are spending almost 50k a year in our network monitor solution, can we go a year without it?"
Quote from: Atrum on January 05, 2015, 10:24:40 AM
*unrelated sidenote @Atrum: FFIX will be remembered as when Squaresoft began their decent into releasing crap games*
Quote from: Seittit on January 05, 2015, 10:50:03 AM
Quote from: Atrum on January 05, 2015, 10:24:40 AM
*unrelated sidenote @Atrum: FFIX will be remembered as when Squaresoft began their decent into releasing crap games*
Ha! I played VII VIII and IX a whole lot. IX was the last for me that really felt like FF. X was OK. I bought X-2, loaded it up and was greeted by a singing pop band Yuna... <sigh>
Here's a question I heard on a podcast recently. What group of IT ppl are the worst? (Networking (no way), Security, VoIP, Server, Desktop etc. ? Thoughts?
Developers.
Yea, Devs are pretty tough. Server guys always seem to throw me off. I've come across some pretty shaky VOIP folks as well.
Quote from: Ironman on January 05, 2015, 03:43:05 PM
Here's a question I heard on a podcast recently. What group of IT ppl are the worst? (Networking (no way), Security, VoIP, Server, Desktop etc. ? Thoughts?
Man, that's tough. I'd agree with Devs. I'd also say security guys (:P). Not necessarily network security guys, but the CISSP types who use a bunch of buzzwords and have zero clue how things really work.
I'm starting to think the ITIL guys are the worst. They seem to live and breath flow carts and diagrams.
Supposedly every change warrants a change ticket, and even if you make a change to fix something that broke then it still requires a change ticket. When I asked, "what if we turn into an environment where the paperwork presents huge roadblocks for even minor, insignificant, non-impaction changes?" I got back: "Then we become an environment with very few outages." Talk about drinking the ITIL Kool-Aid....
Maybe the ITIL statistics are right, and maybe change tickets do help (I believe so to a point)...I still think in a perfect world we should have a middle ground where we can be trusted with minor changes. At the moment, though, I'm not looking to champion that movement.
If you have to do a ticket every time you make a change, then you won't make changes. Increased uptime!
That's exactly what happens IRL. make the process hard and people will avoid it. Forget using initiative and cleaning up little mistakes or suboptimal configurations.
For me, the worst group of people in IT are people that only do enough to get by and have no desire to further themselves within their career. The same individuals that look at you crazy when you bring Cisco Press books to work for reading on your breaks; they also are the people sweating bullets when layoffs are on the outlook (hello fellow energy corridor peeps)
I used to make fun of server people until I became a member of a team with some OUTSTANDING server peeps.
With that said, my experience shows that VoIP peeps are the shakiest in their skill-sets.
Correction: VOIP guys who only know VOIP are shaky. Esp. bastard vendor-specific implementations that even when implementing SIP endpoints throw in non SIP signalling for various events and do not understand standard SIP parameters like a proxy *cough* Cisco *cough* and sell 'SBCs' that don't do endpoint proxy registration and use the world's worst number routing syntax (who the heck thought 'dial peers' was a good idea).
Us router guys who were forced to run call managers (betraying my age there... I lived through the manually patching Win2000 based CallManager 4.1 days) due to the whims of fate, we do understand that the network is our backplane LOL. Also, carrier SIP guys know their stuff.
I'd say in general i agree with devs being the worst. For some reason that probably requires more social demographic analysis, enteprise devs seem to be the bottom of the dev barrel. Their view of ALL infrastructure (not just network, they do the same to server and vmware guys as well) is that its a black box. Which astounds me as you would think a programmer would want to understand how things worked....
Dev: "the network is down my app doesn't work".
Network: "ok what server / where is your app hosted".
Dev: "what?"
Network: "OK fine I'll trawl some firewall logs, what port does your application use?"
Dev: "you tell me, you're the network guy"
:angry:
I have "Fallacies of Distributed Computing" posted in my cube and I refer to it on occasion.
Here are the fallacies:
1. The network is reliable.
2. Latency is zero.
3. Bandwidth is infinite.
4. The network is secure.
5. Topology doesn't change.
6. There is one administrator.
7. Transport cost is zero.
8. The network is homogeneous.
Developers are HANDS DOWN the worst. :choke:
Fun little story,
In our environment we have a VERY VERY OLD oracle 8i 32bit ERP system. Needless to say, CIO says she wants everything 64bit, and we need to get off of the old system and move on past the 90s. 2 Days before the deadline they come to her and say, we cannot get 64bit working, we are going to use a hacked 32bit client to make this work.
here we are on 32bit. :angry:
Or how about the:
we are implementing our F5s in our DMZ environment load-balancing our web servers.
I ask, do the web servers respond in anyway to the clients directly (or grab resources directly), answer is no. 2 months later here we are inserting in x-forward-for so web servers can handle client public IPs..... the list can go on and on.
Quote from: deanwebb on January 06, 2015, 09:06:38 AM
I have "Fallacies of Distributed Computing" posted in my cube and I refer to it on occasion.
Here are the fallacies:
1. The network is reliable.
2. Latency is zero.
3. Bandwidth is infinite.
4. The network is secure.
5. Topology doesn't change.
6. There is one administrator.
7. Transport cost is zero.
8. The network is homogeneous.
9. There is no spoon
Quote from: LynK on January 06, 2015, 10:20:35 AM
I ask, do the web servers respond in anyway to the clients directly (or grab resources directly), answer is no. 2 months later here we are inserting in x-forward-for so web servers can handle client public
I deal with exactly that scenario at least 2-3x per week.
hmmm i have very limited LB experience but in a 'typical' 2 arm deployment doesn't a LB (typically) source NAT the incoming client IP? so the web server should see only the hide IP?
or do you mean that the web app/site uses IP addresses @ layer 7?
Quote from: wintermute000 on January 07, 2015, 05:55:42 AM
hmmm i have very limited LB experience but in a 'typical' 2 arm deployment doesn't a LB (typically) source NAT the incoming client IP? so the web server should see only the hide IP?
or do you mean that the web app/site uses IP addresses @ layer 7?
Typical deployment would use SNAT of some type, changing the source IP.
A surprising amount of applications require the actual client IP address to work properly so you either need to implement X-Forwarded-For or adjust your routing so that SNAT isn't needed.
Developers make me want to :choke:
Maybe I should make that display when someone types in :developers:...
Quote from: deanwebb on January 07, 2015, 12:50:26 PM
Maybe I should make that display when someone types in :developers:...
Don't hesitate. Do.
Done.
:developers:
For those just joining in on the action, type in a colon, then developers, then a closing colon. Then you get the above graphic.
winter & Atrum,
you both are correct. The problem is, they could not clearly define what it is they needed... Once this was accomplished we were able to get them the technologies they needed.
"Hey I know your busy but our day shift unplugged everything and now we can't get our VTC to work. We have a call in half an hour.."
:developers:
I personally feel there are good and bad dudes/ettes in all departments. I just hate IT phonies, just bodies that are taking up a full time position that need their hand held to do anything above basic.
Quote from: jofas88 on January 21, 2015, 09:12:30 AM
I personally feel there are good and bad dudes/ettes in all departments. I just hate IT phonies, just bodies that are taking up a full time position that need their hand held to do anything above basic.
quoted for truth
Things I talk about on a regular basis.
(biggest) I can't get these 2 devices to talk to one another. No I don't know the IP address or ports that are in use.
1.) ICMP type 3 code 3 means the traffic is making it to your server and the server is saying "WTF"
2.) You can't just plug a machine with a static IP address in anywhere and have it work.
3.) No we can't get rid of security because it makes developing more difficult
4.) Of course the firewall is killing your program, you never told me you added that new UDP port.
5.) Your staying on the same vlan, the traffic never even leaves the switch, it's not the firewall that's 4 hops away.
6.) It's your application, I don't know how it works, stop asking me about it.
7.) I appreciate your explanation on how the network works, it was very funny.
8.) No I am not the smartest person in existence, I just know how my stuff works.
Almost forgot the most important.
9.) That IP address isn't in the arp table after I pinged 255.255.255.255, are you sure it's plugged in? Oh it is? You say it now it magically works, I am sooo glad it fixed it's self.
Quote from: dlots on January 21, 2015, 12:14:57 PM
Things I talk about on a regular basis.
(biggest) I can't get these 2 devices to talk to one another. No I don't know the IP address or ports that are in use.
1.) ICMP type 3 code 3 means the traffic is making it to your server and the server is saying "WTF"
2.) You can't just plug a machine with a static IP address in anywhere and have it work.
3.) No we can't get rid of security because it makes developing more difficult
4.) Of course the firewall is killing your program, you never told me you added that new UDP port.
5.) Your staying on the same vlan, the traffic never even leaves the switch, it's not the firewall that's 4 hops away.
6.) It's your application, I don't know how it works, stop asking me about it.
7.) I appreciate your explanation on how the network works, it was very funny.
8.) No I am not the smartest person in existence, I just know how my stuff works.
Almost forgot the most important.
9.) That IP address isn't in the arp table after I pinged 255.255.255.255, are you sure it's plugged in? Oh it is? You say it now it magically works, I am sooo glad it fixed it's self.
:rofl:
I get this one occasionally..
Your device that talks to a server outside of the network was working fine last week? That's funny, because it's not on the domain and it's not in the DHCP allow list. Yes, you have to submit a firewall modification form. No, I don't know the source IP's of this server or what ports it uses.
I get called out this morning to a small client location. I am told it is an emergency and that I must address it personally. The client's onsite tech tells me that their router (Cisco 891) isn't working. Now my first question to any problem is was anything changed in the environment that might have caused the problem. The tech's response was no nothing has changed "we were going along fine and then it just stopped". So my next step since this site had their own tech I asked what they had already done to diagnose the problem. His response was they rebooted everything and that didn't take care of it so he tried to access the router and couldn't and that was when they knew something was wrong with the router.
So I get out my laptop and console into the router (with no problem btw when the onsite guy said the device couldn't be accessed). First thing I noticed is this device looks like it has been defaulted (WTF???). I do a show start and what do ya know the proper config is there but the running config is blank, so I investigate a little further and find that the config register is set to 0x2142 (I know some of you are probably laughing already) as well as the wan port in the start config is set to shutdown. I resolve the issue and get everything back up and running. I then proceed to ask the onsite tech who last touched this device and explained the changes and the situation. He goes "Oh we did have a tech here earlier from the ISP to make a change to our IP Phone system we rent from them. Come to think of it the problem started after he left".
Waiting to hear back from someone at the ISP about who this "tech" was and what exactly he was doing or supposed to be doing.
That is hilarious.
You know you're a networking fiend when "config register is set to 0x2142" is a punchline. :rofl:
I love it when I get the occasional, "I have forgotten more than you know". Or the, "I have over 20 years of experience in IT". :angry: :angry: :angry:
Here is a good instance of what I get.
Hey, we are not able to get to the new web server we published.
I ask, "where did you get the IP from"
"Oh, I just took one...."
I ask "whats the problem"
"oh the web server cannot get to the internet, to the sql server, and cannot be reached internally...."
I say "well now that I know, here you go"
I know more than you've ever forgotten. :lol:
Quote from: LynK on March 03, 2015, 09:59:39 AM
I love it when I get the occasional, "I have forgotten more than you know". Or the, "I have over 20 years of experience in IT". :angry: :angry: :angry:
My experience has taught me that any time you run into anyone with more experience than you, just step back and list. Take the passive approach and just listen to what they have to say. Some times they talk garbage but more often than not there is something you can learn. Also good inquisitive questions help fuel the discussion.
These guys/gals have been in the trenches for a long time and have some valuable knowledge to share as long as you will listen.
Quote
These guys/gals have been in the trenches for a long time and have some valuable knowledge to share as long as you will listen.
I listen, but after you hearing so many inconsistencies... you tend to tune them out. I do respect those who have been in the industry and know their stuff. Not those who get in a comfortable spot, and do just enough so they wont get fired.
There are guys that have been in the trenches... and then there are guys that fell into the trenches and haven't found a way to climb out of them.
Quote from: deanwebb on March 09, 2015, 09:35:19 AM
There are guys that have been in the trenches... and then there are guys that fell into the trenches and haven't found a way to climb out of them.
Yeah - some folks understand that you're never "done" learning.
Some don't, and think things still work the way they did 5-10 years ago.
Ummmm....I just finished explaining how Ethernet works to a coworker that is a "senior" tech with supposedly 20+ years experience. :wall:
In his fairness, he may still be stuck on SNA networks.
I wish that was the case. IMO this guy only has a job because he is life long friends with the majority owner. :whistle:
Quote from: Nerm on April 04, 2015, 11:16:45 AM
Ummmm....I just finished explaining how Ethernet works to a coworker that is a "senior" tech with supposedly 20+ years experience. :wall:
We have a few of those guys here.
E-mail chain with the security engineer of a customer:
Me: Please send me a copy of the routing table from your ASA.
Security Engineer: The ASA is a firewall, it's not a router.
Me: Yes I understand that but I have your configs and I see that you have OSPF configured.
Security Engineer: Ok, give me a few minutes.
*I get an e-mail 10 minutes later and it contained a screenshot of the GUI showing the interfaces and description.*
Me: That just shows the interfaces. I need you to login to the ASA via the CLI and send me the output of "show route".
*I get an e-mail with the screenshot of their entire screen which includes the routing table as well as a Google search for how to view the routing table on an ASA via CLI."
Being professional: how not to do it.
Quote from: FilipiNomad on May 30, 2015, 10:10:13 PM
E-mail chain with the security engineer of a customer:
Me: Please send me a copy of the routing table from your ASA.
Security Engineer: The ASA is a firewall, it's not a router.
Me: Yes I understand that but I have your configs and I see that you have OSPF configured.
Security Engineer: Ok, give me a few minutes.
*I get an e-mail 10 minutes later and it contained a screenshot of the GUI showing the interfaces and description.*
Me: That just shows the interfaces. I need you to login to the ASA via the CLI and send me the output of "show route".
*I get an e-mail with the screenshot of their entire screen which includes the routing table as well as a Google search for how to view the routing table on an ASA via CLI."
Wow! Sadly at this point I give the guy props just for googling how to do it lol.
Wowwww... there's an art to screenshotting that that guy has yet to master... People WILL read the tabs and minimized window titles, inspect desktop icons, systray icons, etc.
Had a guy a few weeks ago come in and he kept mentioning things about something he called the "CLEE" turns out he was referring to the CLI!!!! WTF?
I'm from back in the day... I just call it "opening up a command prompt".
Recently had a meeting with a new client that has 10+ locations. I was called in for network performance issues. Found that these 10+ locations are all VPN back to their HQ which houses their only server doing AD, file sharing, application hosting, etc for all locations. And the single WAN at the HQ is a 10Mb/1Mb SOHO grade connection. The kicker is when I present my findings to them they say their last IT service provider told them when they set all this up several years ago that 1Mb up was all they would ever need.
I am beginning to wonder which is worse; the incompetence in small companies or the bureaucracy in large companies? :wall:
Quote from: Nerm on June 24, 2015, 10:15:59 AM
I am beginning to wonder which is worse; the incompetence in small companies or the bureaucracy in large companies? :wall:
You left out the delicate blend of the two to be found in the midsize companies. :problem?:
Quote from: deanwebb on June 24, 2015, 11:06:12 AM
Quote from: Nerm on June 24, 2015, 10:15:59 AM
I am beginning to wonder which is worse; the incompetence in small companies or the bureaucracy in large companies? :wall:
You left out the delicate blend of the two to be found in the midsize companies. :problem?:
Double Bonus-points for me! I work for a medium sized hospital with about 25 remote clinics all ran independently (outside IT) and we are now part of a large heath system with 5-8 other hospitals. I get the hell that is all three...
Quote from: that1guy15 on June 24, 2015, 11:26:10 AM
Quote from: deanwebb on June 24, 2015, 11:06:12 AM
Quote from: Nerm on June 24, 2015, 10:15:59 AM
I am beginning to wonder which is worse; the incompetence in small companies or the bureaucracy in large companies? :wall:
You left out the delicate blend of the two to be found in the midsize companies. :problem?:
Double Bonus-points for me! I work for a medium sized hospital with about 25 remote clinics all ran independently (outside IT) and we are now part of a large heath system with 5-8 other hospitals. I get the hell that is all three...
Eeeek! I do not envy you at all lol.
Sat in on a meeting with a client the other day planning a wireless project we were doing for them. We were going over the proposed AP placement based on a survey I had done the week before. The VP of the client asks why we need more than a "couple" AP's for a 160,000sq foot facility. It is a rectangular shaped factory and the VP wants to just install 3 AP's down the middle of the plant to save money. I explain to him that for one they need more AP's just for coverage of a facility that large and point out my coverage findings from my survey. I then tell him that besides just the coverage they would need more than 3 AP's just for load-balancing of users alone. He turns and asks why we need to load-balance when they will only have 10-15 people using wireless at any given time.
:wha?:
I was like this is a 160,000sq foot facility, you have over 200 employees, and you expect only 10-15 people will use the wireless at a time? :rofl:
"Why can't you just get one AP and crank it all the way up?"
:facepalm3:
Quote from: Nerm on September 03, 2015, 11:01:08 AM
I was like this is a 160,000sq foot facility, you have over 200 employees, and you expect only 10-15 people will use the wireless at a time? :rofl:
Someone is in for a rude awakening on BYoD to work day (everyday)
I can hear your phone ringing now with IRATE customer wondering why the wifi is soooooo slooooooow.....and constantly dropping..
Reviving topic...
Talked about static routes with my son last night. I'm proud of that boy. He was pleased with how he'd done his CCNA lab on that topic. My wife's eyes kinda glazed over, but, hey, it's a network thang, y'all...
:gangsta:
We just had out first transition meeting with the senior management for our data center closing in 16 months. a cursory overview of services and a DDD of March 2018. we've been attempting to move services for probably the last year or so. But things weren't going as they had hoped.
Ate a big breakfast today because I had a feeling that I wouldn't be able to take a lunch.
I was right.
Quote from: deanwebb on November 09, 2016, 11:34:15 AM
Ate a big breakfast today because I had a feeling that I wouldn't be able to take a lunch.
I was right.
should I have posted in the current frustrations thread? but it's a network thing....they just don't understand.....
should have started like 5 years ago.
I think it's appropriate here, as well. I'm currently frustrated by what they just don't understand.
Quote from: deanwebb on November 09, 2016, 12:44:42 PM
I think it's appropriate here, as well. I'm currently frustrated by what they just don't understand.
Depth, scope, and complexity of services we provide to our customers, that, and the amount of time it takes to coordinate with those customers (ones that don't really understand their network) to migrate their service out of the data center.
If you sat here for a day or two, did some network discovery you be like: :barf:
Same here...
And then there's the guy who is over the technology but who really doesn't quite *get* it. For example, NAC. We are officially notifying sites when we are monitoring and enforcing on their wired network. We're also providing RADIUS services for the wireless - which means automatic monitoring *and* enforcement for those guys with 802.1X. The catch is, we don't notify for those.
So we get a guy at a site with wireless active, wondering why it is that the NAC system account is logging on to endpoints there. He's all :zomgwtfbbq: and calls the guy over the technology who's all like :eek: we're not enforcing in that site yet so he calls me and I say, well, wireless guys get enforced and then he's all
:phone:
because we're supposed to notify a site when we do enforcement, but, I'm all like :matrix: and then he's all whoa, slow down dude give me the :glitch: version and I say that, in order for devices to get on wireless, they pass through 802.1X enforcement, so, yeah, we be all up in their Windows, collecting their datas. No warning necessary, because that's how we roll. Tell the guy bugging you to just 8) and we'll be all cool.
And then he's like, well... can we just turn off the logging on part?
And that's when I'm all :facepalm1: because that's just how the product works, man. It's not a problem in any of the other 200 sites where we're doing the stuff, so, no, we can't. And by can't, I mean we won't.
Now, if he had more technical knowledge about the NAC stuff, he'd have been able to head off the question before it got to me. I know he's got lots of stuff on his plate, I've seen his triple-scheduled calendar... but I do wish that we'd have had some time for him to directly work with the system and not just be responsible over it. Kinda like a guy that's a whiz at keeping a F-16 fighter in good shape may not know what's best for an M-1 tank, if he hasn't got a lot of hands-on experience with it.
I went round and round with our help desk staff because a VIP was having issues copying a multi-gig file from his local workstation to his home drive. They were convinced that because it worked fine while connected to a workbench switch in their office it *had* to be the network. Keep in mind that *THEY* are responsible for everything past the port on the wall, including the patches. We kicked the ticket back and forth a few times as I told them to check/change the patch cables and VOIP phone the computer was daisy chained off of. Finally I broke down and logged into the web interface on the phone. Phone to Switch auto-negotiated at 100Mbps (10/100 PoE switch) and Phone to Computer auto-negotiated at 1Gbps. So I asked them if they had even gone down and looked at the guys setup, changed cables, tried removing the VOIP and checking the connection. Nope, it HAD to be a network issue...
So I got to explain to them how you can't squeeze a 1Gbps through a 100Mbps link. Glazed look, and "So how do we fix it?" I realize that help desk is entry level, but shouldn't they be able to think their way through that one on their own??? Their team lead has been doing help desk here for more than six years and can't find the network adapter speed settings? Or even be bothered to entertain the idea that working on the workbench switch doesn't mean the device is working *correctly*.
Sent from my Nexus 6 using Tapatalk
Oh man, we totally need more people with more skills in this line of work...
@jason,
Please do not take this the wrong way, but why did you wait so long before helping them? I take the approach to train all of our help desk. I teach them speed/duplex, I team them these things because in the long run, it WILL make my life 1000% easier when they know how to check a static IP entry and why a computer will not connect when you move it.
At my old job a lot of the engineers/admins treated our helpdesk like dirt. I was the only one nice to them and trained them. So they took the extra initiative to look at things twice before reaching out to me. It was quite nice.
Agree with LynK. Empower your people and understand you're one big team. Take the time to train/educate (it doesn't take long in small doses), and it pays off in spades.
You are making an assumptions that they want to learn, or care about the customers. That is a bad assumption.
I quit trying to teach/help them because the only thing it got me was them farming more of their work to me. Under our current contract they no longer have a local manager and have quit doing a large majority of their duties. They have completely stopped providing ANY desk side support or troubleshooting. All they do now is re-image peoples drives, and if that doesn't fix it they blame it on the network or the servers.
Sent from my Nexus 6 using Tapatalk
Quote from: jason.copas on December 27, 2016, 10:36:04 AM
You are making an assumptions that they want to learn, or care about the customers. That is a bad assumption.
I quit trying to teach/help them because the only thing it got me was them farming more of their work to me. Under our current contract they no longer have a local manager and have quit doing a large majority of their duties. They have completely stopped providing ANY desk side support or troubleshooting. All they do now is re-image peoples drives, and if that doesn't fix it they blame it on the network or the servers.
Sent from my Nexus 6 using Tapatalk
I hear ya, I can't get our help desk team to tell us BEFORE they swap out someone's PC, instead of after. they just don't grasp how difficult it is to find port after it's been err-disabled via port-secuirty and wonder why the PC isn't working like the old one. if they told us the MAC address of the old computer before they replaced it we could easily find the pc and make port-security adjustments.
I've had similar issues with our printer LCR's, only our guys were tripping the DHCP snooping database instead of port-security.
Sent from my Nexus 6 using Tapatalk
Reading this actually makes me pretty thankful for the helpdesk people we have at my current gig.
Sounds like Jacob would profit from talking to a manager... and if that's already been done, then it may be that they're just waiting for the contract to expire before giving this vendor the heave-ho.
Under the new contract we will all work under the same contract/vendor. And what will be our site lead is as fed up with them as I am.
Sent from my Nexus 6 using Tapatalk
Seriously, 90% of this kind of headache can be resolved by not doing an Operations role.
I know a lot of you guys are in this boat and its not always easy (in real life with mortgages and kids etc.) to change jobs but I'm of the opinion that once you've earned your stripes and your fair share of 2AM war stories its time to move on - just keep looking for anything design/project focused.
Frankly speaking, one of my rules now is no Operations jobs, never again (unless its facebook or some other special cutting edge SDN snowflake, but not the alleged AWS deathmarch (or is that old news now), and being in Oz fat chance anyway LOL). Life's too short to clean up after muppets and be under-appreciated into the bargain.
The second (job) rule I have is that the network / act of net-eng needs to be a profit centre not a cost centre. Amazing how different management views the engineers if they're billing straight to the bottom line or building the thing that generates all the revenue vs some kind of cost that surely we can massively cut by employing a bunch of dirty elbonians. Maybe if its some kind of global mega-corp you could bend this rule, but even then it would weigh heavily in my estimation.
@winter
Implementation jobs also come with their headaches. A few of them include, not being given all the information for a project scope, project creep, and even after you implement you get endless calls because x, y, or z no longer works and it is because of equipment A that you implemented. Fix it all or the company doesn't get paid.
Winter and Lynk bring up some very important points to consider. Jason does, too. What's most important is that we've all got to find what we're comfortable with.
At Global Megacorp, I get to watch as we rake vendors over coals, I see vendors in a knife-fight over who gets a contract, and I see emails every day from sales guys trying to work dat hustle... as I'm on the 2AM escalation call as I discover that this has been an issue for the last 2 and a half months and suddenly NOW it's a firewall problem?
Winter's last point is what I want to address: yes, there is some bending in Global Megacorp. Some. We've had no issue in hiring staff in Eastern Europe, South Asia, Southeast Asia, and Latin America. No more staff in Western Europe and it's very difficult getting clearance to get a guy hired in the USA, but it can still be done. Networks are still a cost center, even if we can show our value in bill-back charges. It's the widget makers that have the final say, which is why I've been in my role for three+ years and we are STILL waiting for one location to take an outage to switch out some gear. We've been trying to get them to upgrade since December 2013... but the gear that is there works, and the staff there have lots of other maintenance tasks to do, so they can't handle the upgrade and there's no travel budget this year, so... no upgrade this year, either.
Great health benefits, compensation has been appropriate so far, and I work with some great people. Didn't get approval for RSA, training so far has been for vendor products I'm working with, not anything in general theory or practice. *That* stuff I have to go get on my own, same for if I want to go after a CISSP.
If I went to another global megacorp, good years mean travel and training, bad years pretty much what I have now, so it's a wash. Once we get into some new product launches, we'll have another good year, money flows, people become the most important resource blah blah blah once again. The only big difference would be in how far I drive and who I work with.
If I went to a VAR, I'd be hella billable... and there would be the firehose of certs and training to keep up with. Travel and training, for sure, but lots of that travel is to client sites. And if clients are totally screwed up, too bad. That's now my fault, if it jeopardizes the project. More money, but with a smaller firm, the health and retirement benefits are smaller. It's overall more compensation and the stresses are different, but still there.
How about a vendor? If an SE, it's about closing the deals and moving that product. Post-sales/TAM is about keeping the product renewed and finding ways to upsell and being a mule to beat when it doesn't work right. Some of those guys have been on the 2AM calls with me.
What about management, then? I'm kind of already doing that now, with directing other guys to help me do what I'm doing. But I don't play golf, I don't like drinking a lot, I wonder if I'd be able to fit in with the social aspect of the role. My brother tried to do that at one place, made some suggestions, everyone looked at him like he was a leprechaun doing a hip-hop routine on the kitchen table, and he got canned after just a few months there. He's now a senior consultant (also a programmer) and is enjoying his current role. He gets in, gets stuff done, gets billed, gets to the next gig.
In all that consideration, if things went south at my job, I'd probably consider most going in as an SE for one of the vendor products I work with. I enjoy them, I really appreciate the tech support I've gotten for them, and I see their value and know how to unlock it. That enables me to push the sale. I can't sell just anything, but I can sell what I can be confident in. I see the headaches the SEs have to deal with - I know I create some of those, myself - and they look like the kinds of headaches I'm good at working through.
But for now, I feel like I have a good balance. It could be better, and I'm hoping we get another guy here in the USA very soon. I'm ready to re-evaluate in March, though.
Quote from: LynK on December 29, 2016, 08:38:29 AM
@winter
Implementation jobs also come with their headaches. A few of them include, not being given all the information for a project scope, project creep, and even after you implement you get endless calls because x, y, or z no longer works and it is because of equipment A that you implemented. Fix it all or the company doesn't get paid.
At least you're in the game and directly determine the outcome. In Ops you just get handed someone else's turd sandwich and told to do great things with it.
Also with larger VAR/consultancies you are protected to a large extent as long as you follow the methodology and get signoffs at every turn, also, there's usually enough management/process layers to make sure that it is done correctly and no individual has to be left holding the bag. Covering your backside starts becoming second nature after awhile (in a good way as in getting people to agree to things in writing!!!). e.g. scope creep - if you did your statement of works correctly then it shoudl be clear what is a variation and what is not - either the sales guy eats it in margin for brownie points, or you issue a variation, either way it takes as long as it takes.
Deanwebb, in my line of work, pretty much 90% of people leaving go straight to vendor-land, esp. as its usually a nudge nudge wink wink arrangement with one of our gold/platinum partners.
Quote from: wintermute000 on December 29, 2016, 04:24:06 PM
Deanwebb, in my line of work, pretty much 90% of people leaving go straight to vendor-land
This is the truth. I know a lot of friends of mine who worked with VARs, they liked it but vendor land is VERY cushy. So off they went.
Quote from: LynK on December 30, 2016, 08:45:21 AM
Quote from: wintermute000 on December 29, 2016, 04:24:06 PM
Deanwebb, in my line of work, pretty much 90% of people leaving go straight to vendor-land
This is the truth. I know a lot of friends of mine who worked with VARs, they liked it but vendor land is VERY cushy. So off they went.
My former team lead went to a vendor as an SE. He's smiling, for sure. Every day, he's smiling.
Quote from: deanwebb on December 30, 2016, 09:47:49 AM
Quote from: LynK on December 30, 2016, 08:45:21 AM
Quote from: wintermute000 on December 29, 2016, 04:24:06 PM
Deanwebb, in my line of work, pretty much 90% of people leaving go straight to vendor-land
This is the truth. I know a lot of friends of mine who worked with VARs, they liked it but vendor land is VERY cushy. So off they went.
My former team lead went to a vendor as an SE. He's smiling, for sure. Every day, he's smiling.
I know I am... :problem?:
Quote from: AspiringNetworker on December 30, 2016, 11:37:55 AM
I know I am... :problem?:
Lol... for sure, I want to finish a successful implementation of $VENDOR at Global Megacorp before sailing off to work for $VENDOR. If I can say not just "Sure, it's scalable!" but, "Scalable? I made it work for Global Megacorp in over 200 sites, with over 150,000 employees. Sure, it's scalable!" that's the kind of thing that managers, executives, and board members want to be able to hear.
I feel that if I bail and go to $VENDOR now, it could mess up their relation with Global Megacorp, and that wouldn't be good for my career path with either Global Megacorp or $VENDOR.
You'd want an under the table agreement first between your employer and vendor. That's how most of these things go down.
Quote from: wintermute000 on December 31, 2016, 04:17:27 PM
You'd want an under the table agreement first between your employer and vendor. That's how most of these things go down.
Duly noted... which does one generally initiate the discussion of such with? The vendor or the employer?
TBH I'm not entirely sure. I think usually the vendor puts the feelers out after sounding out the individual. With these kinds of things, we're talking Gold/Platinum partners etc. so both mgt and sales teams are very close and have usually worked together for years and done millions of $$$ of deals together so its typically an organic process.
This is esp true because you can usually see it coming a million miles away e.g. hey surprise, the guy that's done nothing but Fortinet installs for the last 4 years has gone to... Fortinet. etc.
So you can see that in this kind of situation it makes sense to keep everyone on-board. The channel is extremely incestuous
That's what I thought. Thanks.
Tech: Can you add 172.35.10.0/24 into our global WAN?
Me: That isn't rfc1918 does your location own that subnet?
Tech: What? It is in the 172.16.0.0/12 range.
Me: Go home. You're drunk.
Quote from: Nerm on January 05, 2017, 08:29:22 AM
Tech: Can you add 172.35.10.0/24 into our global WAN?
Me: That isn't rfc1918 does your location own that subnet?
Tech: What? It is in the 172.16.0.0/12 range.
Me: Go home. You're drunk.
:haha4:
Today's big crisis:
WE CAN'T REACH THE SERVER!!! ZOMGWTFBBQ IT MUST BE NAC!!!
No, it's not the NAC. We're only monitoring the traffic on that LAN.
THEN IT IS FIREWALL!!! TURN OFF ALL THE FIREWALL!!!
No, the path does not go through a firewall.
IT IS IPS!!! YOU MUST DEACTIVATE THE IPS AND TOSS IT INTO THE SEA!!!
Hold on, is this a physical or virtual server? They go through different segments on the DC IPS.
PHYSICAL OR VIRTUAL? WE SHALL CHECK THE THINGS...
And what did they discover?
***the reason the virtual server was unreachable was because it had not yet been installed yet***
:mssql:
:notthefirewall: :ivan:
Quote from: Nerm on April 07, 2017, 12:39:53 PM
:notthefirewall:
I had to send one of those out today, but had to modify it to change 'blocking' to 'dropping'
@dean, when are you actually going to enforce NAC?
Had my own two issues today (both my fault). New palo alto firewalls only getting 97mbps down/94 mbps up. I swap laptops to see if it is my machine. Same thing. I call TAC and work with them for 4 hours. I go take my laptop unplug the internet uplink from the fw to the interwebs into my laptop... same speed. HUH?!?!
(LIGHTBULB) I had the speed hard set on BOTH LAPTOPS because I was messing with old firewalls. *(kill me)*
Call TAC case in because new VPN can be reached internally, but internals cannot reach external VPN user (LIGHTBULB = WINDOWS FIREWALL) *(KILL ME)*
:rage:
Enforcing NAC in three of our sites, currently. I'd like to have all the rest of the system installed as designed before we turn on any more. No budget, no progress.
And now my firewall management project without a project is being looked at by two different managers, each owning half of the system.
If I don't put cover sheets on my TPS reports, I'm going to never hear the end of it.
:gangsta:
Whenever we turn on an NMAP scan against our HP printers, they have issues.
Sent some network traces to HP and asked what can they do about it.
HP guy sends back a big mansplaining thing about how to do a network capture properly, even shares his whitepaper on the art of Wiresharking and then asks...
"By the way, what is NMAP? Is that some kind of freeware thing?"
:ivan:
Quote from: deanwebb on April 14, 2017, 08:50:00 AM
Whenever we turn on an NMAP scan against our HP printers, they have issues.
Sent some network traces to HP and asked what can they do about it.
HP guy sends back a big mansplaining thing about how to do a network capture properly, even shares his whitepaper on the art of Wiresharking and then asks...
"By the way, what is NMAP? Is that some kind of freeware thing?"
Sounds like its your turn to return the favour.
Quote from: ristau5741 on April 14, 2017, 10:37:31 AM
Sounds like its your turn to return the favour.
I gave him a link. He downloaded the tool and couldn't reproduce the error, so he sent an email basically asking if it was network stuff actually causing the printers to fail and how it couldn't possibly be network stuff because HP is the diggity-bomb and and and...
I replied by explaining how, with NMAP active, the printers fail. When NMAP is inactive, the printers do not fail. Therefore, it may be wise to find out what printers we have with the worst problems in this area and set one up in your lab just like it and fire away the NMAP scan.
HP came back with, "Well, maybe it's port 9100. Stop scanning that and see if that fixes things." :problem?:
No, low ports knock them out, as well... can you get the repro, pleeeeeeeeeeeeeeeeaaaaaase?
This is going to be a very long email back and forth...
:ckfacepalm:
https://www.reddit.com/r/networking/comments/6wor0u/converting_to_a_flat_network/
:mrgreen:
I just saw that... poor guy.
Time for him to quit as he is about to get fired.
That just seems mean. We need you to murder this puppy you have been caring for and nurturing for the last X years. After that you will be fired.
Its always the server guys against the networking guys. I've been working in networking way too long.
But I would have to say the all time "server isn't working issue" top reason is:
No default gateway set.
So whenever a server dude asks me why his server isn't working, can't reach the network, etc. My first question is whats your IP, subnet, default gateway.
That usually fixes like 90% of server issues.
Quote from: Motley01 on September 03, 2017, 11:54:35 PM
Its always the server guys against the networking guys. I've been working in networking way too long.
But I would have to say the all time "server isn't working issue" top reason is:
No default gateway set.
So whenever a server dude asks me why his server isn't working, can't reach the network, etc. My first question is whats your IP, subnet, default gateway.
That usually fixes like 90% of server issues.
If it gets to me, it's because he thinks maybe NAC or the firewall is blocking his box... then I ask, "Can you ping the default gateway?"
If he asks, "What's a default gateway?", we got him cold. :smug: