Fun Project: Building Out a Global Network

[deanwebb]

Hey, I saw Elfiq in the list! I love their stuff! Great tech support, as well.

OK, so SD-WAN all around the world.

Except in China. We can't fling data to the Internet in China, since the gov't there doesn't like that. We can send it across an MPLS network, out Hong Kong, and then to Hyderabad IT Center / Singapore major brokerage center. How do we get that to play well with SD-WAN?

[SimonV]

Please tell us more about the applications they are using?

Is e-mail hosted in the cloud or in the central datacenters?
Are we using Skype to communicate, with all the fancy video and screensharing features?
Still using fileshares on local file servers or is everything hosted on local/cloud Sharepoint?

Other stuff to consider:

By the way, I am volunteering for the vendor dinners once they get wind of this project!

[icecream-guy]

Please tell us more about the applications they are using?

Is e-mail hosted in the cloud or in the central datacenters?  O365 in Azure
Are we using Skype to communicate, with all the fancy video and screensharing features?  Yes , and integrated with the VOIP soft phones
Still using fileshares on local file servers or is everything hosted on local/cloud Sharepoint? A mix, local file servers, with partially completed migration to cloud in AWS

[deanwebb]

Please tell us more about the applications they are using?

Is e-mail hosted in the cloud or in the central datacenters?  O365 in Azure
Are we using Skype to communicate, with all the fancy video and screensharing features?  Yes , and integrated with the VOIP soft phones
Still using fileshares on local file servers or is everything hosted on local/cloud Sharepoint? A mix, local file servers, with partially completed migration to cloud in AWS

I'll add that there are big concerns from the security team about PII in the cloud.

To be fair, the security team always has big concerns about *everything*, but we think they're actually justified in this case. So much so as to raise a formal risk assessment line-item about it. Attached is an article about how an AWS guy allowed "All Authenticated Users" to access data on every US household from the US Census, hosted by Experian. Yes, that meant *any* AWS user got to download all 37GB of the database before it got secured. Over 500 datapoints on every household, as well.

Therefore, Security is keeping a close watch on how we handle encrypting data in motion.

[wintermute000]

Never heard of Viptela,   you had me interested until I saw this....

You should do some more homework. They were the hottest thing in SD-WAN, so hot that 6 months ago CSCO forked out ~650 million and acquired them. So yeah they're Cisco now, but they were acquired precisely because they were demolishing IWAN. Go check out some older packet pushers podcasts if you don't believe me.


12 months ago I was the biggest Arista/Viptela/NSX fanboy out there, but the 900 pound gorilla is turning around slowly....

Look into the technology, its basically a cross between LISP (routing based on locations not destination IP, and hey there's cloud management so the traditional LISP server chicken-and-egg issues go away) and MPLS (label separation) with mandatory FVRF, built in multi-tenancy, BFD monitoring of every point to point leg and using the cloud to abstract away the traditional computational scalability problem of IPSEC phase 2. The MPLS style labelling also gives you built in multi-tenancy (VRFs without having to worry about per-VRF routing config) as well as the ability to vary your topology per 'VRF' (e.g. have a multi-point corporate VRF and a hub-and-spoke PoS network both over the same single overlay).


Its incredibly elegant and foundationally streets ahead of everyone else (who are all basically doing PBR using a DMVPN type overlay to flatten the next hop topology). There's no other SD-WAN vendor with their level of maturity and footprint - 6000 site reference customers (!!!), multiple (>2) US banks etc. Their devs are all former CSCO/JNPR/ALU routing devs who got sick of being handcuffed (I heard that one of the key guys was the key guy behind the original DMVPN code stack). They're not gussied up SMB class crap or WANop / FW guys putting lipstick on a pig (*cough Ri***bed cough *Fort**et**)

Protip: in 6 months you'll be able to buy an ISR with a Viptela component on it, and in 12 months there will be complete migration of the full Viptela stack onto ISR hardware. IWAN is still being supported, but the writing is clearly on the wall. IWANv3 (using, ironically, LISP) is scrapped, no more feature development, its dead, Viptela killed it. They're also going to roll the management plane (currently cloud instances out of AWS) into the SD-Access controller (DNA center i.e. the replacement for APIC-EM).


re: China, no problem, we'll just have to run a traditional IP-VPN to the china site, Viptela can happily overlay over any transport whether private or public. You just need to make sure there is an underlay default route (internet) accessibility so the China CPE can hit the cloud controller. No you don't get the SD-WAN fancy load sharing/failover features (unless you want multiple WAN transports), but it still abstracts away your routing topology issues.

[icecream-guy]

Here are some of the corporate locations

Code Select Expand


City Nation
Chongqing China
Shanghai China
Delhi   India
Beijing   China
Mumbai   India
Lagos   Nigeria
Karachi   Pakistan
Dhaka   Bangladesh
Guangzhou China
Istanbul Turkey
Tokyo   Japan
Bengaluru India
Moscow   Russia
São Paulo Brazil
Lahore   Pakistan
Cairo   Egypt
Kinshasa DR Congo
Jakarta   Indonesia
Seoul   Korea, South
Wenzhou   China
Mexico City Mexico
Lima   Peru
London   United Kingdom
Xi'an   China
Hyderabad India
Chennai   India
New York City United States
Shenzhen China
Bangkok   Thailand
Suzhou   China
Nanjing   China
Dongguan China
Tehran   Iran
Quanzhou China
Shenyang China
Bogotá   Colombia
Ho Chi Minh CityVietnam
Hong Kong China
Baghdad   Iraq
Fuzhou   China
Changsha China
Wuhan   China
Tianjin   China
Hanoi   Vietnam
Rio de Janeiro Brazil
Qingdao   China
Foshan   China
Zunyi   China
Santiago Chile
Riyadh   Saudi Arabia
Ahmedabad India
Singapore Singapore
Shantou   China
Ankara   Turkey
Yangon   Myanmar
Saint PetersburgRussia
Casablanca Morocco
Abidjan   Ivory Coast
Chengdu   China
Alexandria Egypt
Kolkata   India
Surat   India
Johannesburg South Africa
Dar es Salaam Tanzania
Shijiazhuang China
Harbin   China
Giza   Egypt
İzmir   Turkey
Zhengzhou China
New Taipei City Taiwan
Los Angeles United States
Changchun China
Cape Town South Africa
Yokohama Japan
Khartoum Sudan
Guayaquil Ecuador
Hangzhou China
Xiamen   China
Berlin   Germany
Busan   Korea, South
Ningbo   China
Jeddah   Saudi Arabia
Durban   South Africa
Algiers   Algeria
Kabul   Afghanistan
Hefei   China
Mashhad   Iran
Pyongyang Korea, North
Madrid   Spain
Faisalabad Pakistan
Baku   Azerbaijan
Tangshan China
Ekurhuleni South Africa
Nairobi   Kenya
Zhongshan China
Pune   India
Addis Ababa Ethiopia
Jaipur   India
Buenos Aires Argentina
Incheon  Korea, South
Quezon City Philippines
Kiev   Ukraine
Salvador Brazil
Rome   Italy
Dubai   United Arab Emirates
Luanda   Angola
Lucknow   India
Kaohsiung Taiwan
Kanpur   India
Surabaya Indonesia
Taichung Taiwan
Basra   Iraq
Toronto   Canada
Taipei   Taiwan
Chicago   United States
Osaka   Japan
Quito   Ecuador
Chaozhou China
Fortaleza Brazil
Chittagong Bangladesh
Bandung   Indonesia
Managua   Nicaragua
Brasília Brazil
Belo Horizonte Brazil
Daegu   Korea, South
Houston   United States
Douala   Cameroon
Medellin Colombia
Yaoundé   Cameroon
Nagpur   India
Cali   Colombia
Tashkent Uzbekistan
Nagoya Japan
Isfahan   Iran
Phnom Penh Cambodia
Kochi   India
Paris   France
Ouagadougou Burkina Faso
Lanzhou   China
Kano   Nigeria
Dalian   China
Guatemala City Guatemala
Havana   Cuba
Rawalpindi Pakistan
Medan   Indonesia
Accra   Ghana
Visakhapatnam India
Gujranwala Pakistan
Jinan   China
Karaj   Iran
Peshawar Pakistan
Minsk   Belarus
Caracas   Venezuela
Sana'a   Yemen
Sapporo   Japan
Tainan   Taiwan
Bucharest Romania
Curitiba Brazil
Shiraz   Iran
Vienna   Austria
Brazzaville Congo Republic
Bhopal   India
Hamburg   Germany
Manila   Philippines
Kuala Lumpur  Malaysia
Maputo   Mozambique
Budapest  Hungary
Warsaw   Poland
Lusaka   Zambia
Kathmandu    Nepal
Tabriz   Iran
Hyderabad  Pakistan
Palembang  Indonesia
Almaty   Kazakhstan
Tijuana   Mexico
Patna   India
Montreal  Canada
Davao City  Philippines
Harare   Zimbabwe
Barcelona  Spain
Maracaibo  Venezuela
Caloocan  Philippines
Philadelphia  United States
Novosibirsk  Russia
Phoenix   United States
Bulawayo  Zimbabwe
Oran   Algeria
Semarang  Indonesia
Recife   Brazil
Kobe   Japan
Daejeon   Korea, South
Kampala   Uganda
Kawasaki  Japan
Guadalajara  Mexico
Auckland  New Zealand
Vijayawada  India
Fukuoka   Japan
Kwangju   Korea, South
Porto Alegre  Brazil
Kyoto   Japan
San Antonio  United States
Santa Cruz de la Sierra  Bolivia
Munich   Germany
Kharkiv  Ukraine
Yekaterinburg  Russia
San Diego  United States
Barranquilla  Colombia
Milan   Italy
Ibadan   Nigeria
Makassar  Indonesia
Córdoba   Argentina
Prague   Czech Republic
Mandalay  Myanmar
Dallas   United States
Montevideo  Uruguay
Nizhny Novgorod  Russia
Abuja   Nigeria
Calgary   Canada
Saitama   Japan
Hiroshima  Japan
Rosario   Argentina
Brisbane  Australia
Belgrade  Serbia
Campinas  Brazil
Ulsan   Korea, South
Omsk   Russia
Dakar   Senegal
Abu Dhabi  United Arab Emirates
Monterrey  Mexico
Tripoli   Libya
Rostov-on-Don  Russia
T'bilisi  Georgia
Fez   Morocco
Birmingham  United Kingdom
Yerevan   Armenia
Cologne   Germany
Tunis   Tunisia
Islamabad  Pakistan


[SimonV]

Great, we could use the UN/LOCODE standards to determine site IDs and hostnames - two letters for country and three letters for city.

http://www.unece.org/cefact/locode/service/location

I don't see any offices in Ireland or Luxembourg so I suppose this enterprise is paying their taxes properly?

[deanwebb]

Great, we could use the UN/LOCODE standards to determine site IDs and hostnames - two letters for country and three letters for city.

http://www.unece.org/cefact/locode/service/location

I don't see any offices in Ireland or Luxembourg so I suppose this enterprise is paying their taxes properly?

What do you think the Libyan office is for?

For using the Nation/City combo, that's part of just about every company's naming standard. This would mean using our naming convention as a network access criteria is useless. Therefore, we're going to install certificates on everything. EVERYTHING. The Windows AD CA is up and running and ready to issue non-interactive certificates for all our devices.

No cert, no network access, that's the final state for all devices that can support a cert install. For those that cannot, we need to define ACLs that limit their connectivity to the rest of the network. We can't put everything behind a firewall, but we can put a lot of stuff behind an ACL.

Speaking of ACLs and country codes... IP address management... Yes, we're going into a full conversion of everything, but the database guys and developers are really digging in their heels in resisting switching their thousands of servers from using public addresses to an RFC 1918 address space. And don't even get them started about IPv6...

[icecream-guy]

with all those certs, the CA private keys are really really important, will have to devise a way to protect the CA server, without hindering production, eg. daily cert generation.

GL with IPv4 in China... and probably all the Asia-pacific area.

[deanwebb]

with all those certs, the CA private keys are really really important, will have to devise a way to protect the CA server, without hindering production, eg. daily cert generation.

One of the best protections for the CA server is to have secondary servers set up that can do validation, then take the primary server off the network. Some orgs even go so far as to shut it down and to put it into a locked safe.

[icecream-guy]

project funding must have dried up.

[deanwebb]

project funding must have dried up.

That happens, but good news is that we just got our Q1 budget approved. And some server guy has an idea about using NAT for the data center so that the servers keep their public IP addresses, but we do one-to-one static NAT to an RFC 1918 address... What's a nice way of saying

[SimonV]

Aren't we using loadbalancers in the perimeter?

[Otanx]

Just had a presentation on Viptella, and it looks pretty cool. I thought the piggybacking on the BFD packets for IPSLA was a neat trick. Would be a good choice for this project.

-Otanx

[deanwebb]

Aren't we using loadbalancers in the perimeter?

Yes, but management is concerned about not overloading the loadbalancers... so they put all they loadbalancers behind other loadbalancers.