NMAP and HP Printers

[deanwebb]

Got a quick request to anyone with an HP printer and either NMAP or Zenmap (the Windows version of NMAP).

0. Send a print job to an HP printer. Note the speed of that job.
1. Do a series of scans on the HP printer on ports 80 and 443. About 5-10 in a row.
2. While doing those scans, resend the print job from (0) to the HP printer. Note the speed of that job. Was it slower than when it was sent in step (0)?
3. Let me know your results, including the make/model/firmware of your HP printer. Also let me know if it required a reboot after you ran the scans.

WARNING: Doing this test may require that you reboot your HP printer to get it back to normal operation.

We've seen this issue with ColorJet MFPs and JetDirects. The MFPs were most likely to need a reboot.

I've been asking HP to do this test for the last few weeks and they haven't done any testing on their end to try to reproduce the error. If I can get a repro of the error outside my environment, then that can be a big help in getting them to get their act together. If you do reproduce the error, I will request permission to use your company name in my discussions with HP, as in:

"I was discussing this with some colleagues at $OTHER_COMPANY, and they said that they also saw something like this."

And, if any of you are wondering... this is the same guy that claimed to be a WireShark expert that knew nothing at all about NMAP until three weeks ago, when I mentioned it to him.

[Ctrl Z]

I'm curious. Why are you fighting this battle with them?

[deanwebb]

Because my NAC product does NMAP scans, and it keeps knocking our HP printers out of action. I'm the NAC guy, so I get involved in the calls because my stuff breaks things.

Looks like we need to also include the software running on the print server and the printer itself, as a bog standard HP printer doesn't go down with an NMAP scan.

[Ctrl Z]

I do the same thing with printers, including HP printers. Never noticed nor heard of any issues related to it though. But I'm not scanning the printers very often either.

[deanwebb]

Our VA scanner rips 'em a new one every week, right on schedule. Our sites can prepare for that. When NAC started doing hourly scans, they cried uncle.

[Ctrl Z]

Do the scans seem to have a cumulative effect on the printers? Meaning, is the problem only during an NMAP scan or does it persist even after the scan is complete?

[deanwebb]

Both. The more we scan, the more printers that fall over dead.

[LynK]

what is the need for hourly intense port scans? you are asking for issues.

[deanwebb]

what is the need for hourly intense port scans? you are asking for issues.

I'll explain, but that made me think of how a modified "Breaking Bad" quote explains my job and my mindset. The dialogue isn't directed at you, just want to be clear on that. I imagine it delivered more to someone in HR or a VP of something, preferably named Skyler...

Who are you talking to right now? Who is it you think you see? Do you know how much I make a year? I mean, seriously, the HR website is pretty hard to navigate. I don't know, myself. Do you know what would happen if I suddenly decided to stop going into work? A major multinational goes belly up. Disappears! It ceases to exist without me. No, you clearly don't know who you're talking to, so let me clue you in. I am not in danger, Skyler. I am the danger. A guy connects to the LAN and gets put in a remediation VLAN and you think that of me? No. *I* am the one who NACs!

If a device responds in a manner consistent with a certain vendor's printer line in one scan and then starts responding in a manner inconsistent with a certain vendor's printer line in the next scan, then ZOMG H4X are going on.

To pin down a vendor, scanners will send badly-formed packets or incomplete requests to get vendor-specific error messages. This involves multiple packets sent to HTTP and HTTPS ports.

To date, all our other vendors' printers aren't having this issue and HP can't reproduce it in their lab, which means we have some squirrelly thing going on in how we've configured our printers that makes them vulnerable to this sort of scan. The scan itself is not an intense one. It's a port sweep with follow-up probes on ports 80 and 443.

[icecream-guy]

Move all your printers into dedicated 'printer' vlan's this should isolate your problems.

seroiusly though, if you you could probably modify the sans so the printer don't break. or at least turn knobs on and off to try to identify exactly what the problems is.

[deanwebb]

Move all your printers into dedicated 'printer' vlan's this should isolate your problems.

seroiusly though, if you you could probably modify the sans so the printer don't break. or at least turn knobs on and off to try to identify exactly what the problems is.

First one is an option for the big sites, not the sites with everything, even wireless, on a /26. We looked at that.

Since the scans are what determines friend from foe, they're not getting modified. HP is sending a guy out to one of our sites to look things over as I blow up printers. My hope is that he says something magical like, "firmware upgrade" or "turn off that feature, you don't need it, anyway" or something like that.

[Ctrl Z]

You could always limit the attack footprint of the printer by only allowing HTTP/HTTPS access from printer admins and the NAC NMAP scanner, etc. Perhaps that would allow you to scan less often.

Question about your point where a device/user gains certain network authorization based off of initial device profile then somehow changing it's device profile while still keeping the original network access. I guess I'm failing to see a real world scenario where a NAC product wouldn't already be handling the scenario without re-profiling the device so frequently. Could you elaborate?

[deanwebb]

NAC has two functions, if it's working properly. Guard at the gate and bouncer. Most folks think guard at the gate and they're done. But the bouncer function is the most important one.

Say the guard at the gate lets in Brad Pitt, but it turns out once Brad gets inside, he turns out to be a chimp with a machine gun in disguise. As the bullets fly, the guard at the gate is still checking who's on the list. He's not touching that noise on the inside.

So, for the job of the bouncer to work, he's got to be able to check on the status of clients and then take appropriate action when they change.

Looking at the printers, the way they work is for users to hit them via a web interface to set up their print jobs. Port 80 and 443 aren't getting locked down anytime soon. This is the world of apps and developers, and they want all the ports open, all the time.

So, we scan. Let's say some guy has programmed a raspberry pi to have the same MAC address as a printer in some closet somewhere that's never used. Criminals love areas where nobody goes and all those printers, phones, and other junk plugged in that's ignored are the digital equivalents of abandoned warehouses. Anyway, let's scan once per week in a predictable pattern. Attacker knows to swap out for a little while each week, never gets noticed. Scan once per week, randomly, guy can roll his dice or just hope that nobody notices the swap. If they're only scanning once per week, maybe they have a SOC that just grunts when it gets another alert in a stream of millions...

But if we scan all the time, we have a better chance of noticing the swap when that raspberry pi doesn't respond like a printer when we poke it in the eye, ever so gently. We make a rule to alert on a device function change and then put the device into a remediation VLAN and now that r-pi is doing long-term surveillance on nothing of importance.

Similar logic applies to Windows devices that get online but then suddenly fire up a local Kali Linux VM. Or other stuff.

Security is not about blocking ports and then going home to sleep. It's about trying to be an audience for a magic trick that the performer would rather keep totally secret.

If all we do is a MAC bypass, that's a blank check made out to cash and handed to the criminal. *any* device with that MAC address gets on and stays on. If we do 802.1X for the wired LAN, I can show a device that will ride on another device's legitimate network admission by leeching off the line and duplicating its MAC address to match the legit device. Spotting *those* guys is now not a matter of scanning, although that can catch them sometimes, but keeping an eye on inbound/outbound traffic from that host. Again, though, there needs to be integration with the NAC system so logic can be written to shut down unauthorized access from an endpoint while still allowing legitimate network usage.

[Ctrl Z]

All concepts I'm very familiar with, just never heard of any place that re-profiled devices anywhere near as often as you're doing.

Personally, I prefer to approach it by limiting access to/from the connected endpoint with the idea of minimizing the impact of any endpoints that might get compromised/spoofed/disgruntled employee/etc. I'm also combining that with netflow data to try and identify suspicious behavior. I say try because as always, these things take a massive amount of initial tuning and constant on-going tuning to minimize the number of false positives and false negatives.

I always try to approach NAC and network security in general with the perspective of "how would I protect this network from myself, if I got fired tomorrow" then try to protect against that.

[deanwebb]

It's just once an hour, we're not asking for much.

And if I had to protect the network against myself, I would need to kick off a MASSIVE social engineering and phishing awareness program.