Meraki: 5 API Calls per Second

Started by deanwebb, September 24, 2019, 11:00:07 AM

Previous topic - Next topic

deanwebb

Hitting this limitation where we learn that in a Meraki Organization, there is a hard limit of five API calls/second, total. That's not per account or per network or per user or per anything other than the org, and you get five. Figure out who gets to use what, but you get five/sec and then error 429 messages when you exceed that number.

Can you batch calls? Yes on puts, no on gets. Can you query the org for info instead of the network? Yes, but it only reduces the total calls by a small amount.

Mist is even tighter with their API calls, about 1.4 per second...

Customer is hitting that API limit on Meraki really hard right now and Meraki is telling them to split out their orgs and make a bunch of new ones.

Customer is NOT happy with that recommendation...

:caine:
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

Dieselboy

Thats a rubbish recommendation. That would begin to exponentially grow the amount of work required to manage the company's devices. Did support ask if they can close the case after sending that over?  :evil:

I would be looking to see if they can code in to respond to a 429 code with a backoff and re-try. Something like this:

1. send GET
2. 429 response received
3. retry GET
4. 429 response
5. Backoff random time (but a short time like 2 seconds or less)
6. retry GET
7. 429
7. Backoff longer random time, say 5 seconds
8. retry GET
9. success

Basically, take the 439 code as a message to mean try again in a little while. Sounds like they are receiving the 429 as a response and as their system doesnt know what to do with it, it's taking it as "this is the response. We're now finished".

deanwebb

Actually, the backoff timer was 30 seconds, so the calls would stack up and age out... not good when you're trying to NAC stuff.

We set the timer to 0. Got a lot more errors, but also got a lot more calls serviced... we just hammer away at the Meraki and figured that wasn't a good thing, long term and in production, so we put the timer up a bit. Still not happy with the results...
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

Dieselboy

In that case I'd just hammer away until you get a response other than 429. Sounds like they're policing the requests, rather than it's a case of that is all that their systems are capable of handling.

deanwebb

Quote from: Dieselboy on September 26, 2019, 09:32:53 PM
In that case I'd just hammer away until you get a response other than 429. Sounds like they're policing the requests, rather than it's a case of that is all that their systems are capable of handling.

Exactly. It's policing, and it makes no sense to us why there's a limit at 5 instead of, say, 20 or 8 or 37...
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.