VPN help needed

[heath]

I know the answer is simple and I've just been looking at it to long for it to jump out at me. Or, more likely, I'm just ignorant.

I'm setting up a remote access VPN for a branch location.  At that branch office, they have internet through a cable modem - so a dynamic IP address, not static.  The firewall I have for the branch location is an ASA 5506W-X.  It's running the latest software - 9.5.2 (asa952-lfbff-k8.SPA).  (I do want the wireless access point on this ASA functional and traffic allowed through the VPN tunnel, but that's a secondary concern.)

The firewall at the home office is an ASA 5506-X.  Running the same software version.  It has a static outside address. 

I can get the tunnel up.  The branch ASA encrypts the packets and sends it to the head end:
      #pkts encaps: 34, #pkts encrypt: 34, #pkts digest: 34
      #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0


The home office ASA receives and decrypts the peackets, but doesn't send anything back:
      #pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
      #pkts decaps: 34, #pkts decrypt: 34, #pkts verify: 34



Relavent Branch 5506W-X Config:

Code Select Expand

interface GigabitEthernet1/1
nameif outside
security-level 0
ip address dhcp setroute
!
interface GigabitEthernet1/2
nameif inside
security-level 100
ip address 192.168.11.1 255.255.255.0
!
interface GigabitEthernet1/9
nameif wifi
security-level 100
ip address 192.168.10.1 255.255.255.0
!
object network obj_any
subnet 0.0.0.0 0.0.0.0
object-group network Branch-Networks
network-object 192.168.11.0 255.255.255.0
network-object 192.168.10.0 255.255.255.0
object-group network Home-Networks
network-object 10.0.0.0 255.0.0.0
access-list outside_cryptomap extended permit ip object-group Branch-Networks object-group Home-Networks
pager lines 24
logging enable
logging asdm informational
mtu outside 1500
mtu inside 1500
mtu wifi 1500
icmp unreachable rate-limit 1 burst-size 1
asdm history enable
arp timeout 14400
no arp permit-nonconnected
nat (inside,outside) source static Branch-Networks Branch-Networks destination static Home-Networks Home-Networks
!
object network obj_any
nat (any,outside) dynamic interface
!
nat (inside,outside) after-auto source dynamic any interface
crypto ipsec ikev2 ipsec-proposal AES256
protocol esp encryption aes-256
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal AES192
protocol esp encryption aes-192
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal AES
protocol esp encryption aes
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal 3DES
protocol esp encryption 3des
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal DES
protocol esp encryption des
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal Site2Site
protocol esp encryption aes-256
protocol esp integrity sha-1
crypto ipsec security-association pmtu-aging infinite
crypto map outside_map 1 match address outside_cryptomap
crypto map outside_map 1 set pfs group5
crypto map outside_map 1 set peer xxx.xxx.xxx.xxx
crypto map outside_map 1 set ikev1 phase1-mode aggressive group5
crypto map outside_map 1 set ikev2 ipsec-proposal Site2Site
crypto map outside_map 1 set reverse-route
crypto map outside_map interface outside
crypto ca trustpool policy
crypto isakmp identity key-id Branch_Map
crypto isakmp disconnect-notify
crypto ikev2 policy 2
encryption aes-256
integrity sha512
group 24
prf sha512
lifetime seconds 86400
crypto ikev2 policy 3
encryption aes-256
integrity sha
group 5 2
prf sha
lifetime seconds 86400
crypto ikev2 policy 10
encryption aes-192
integrity sha
group 5 2
prf sha
lifetime seconds 86400
crypto ikev2 policy 20
encryption aes
integrity sha
group 5 2
prf sha
lifetime seconds 86400
crypto ikev2 policy 30
encryption 3des
integrity sha
group 5 2
prf sha
lifetime seconds 86400
crypto ikev2 policy 40
encryption des
integrity sha
group 5 2
prf sha
lifetime seconds 86400
crypto ikev2 enable outside

group-policy VPN_Policy internal
group-policy VPN_Policy attributes
vpn-tunnel-protocol ikev2
dynamic-access-policy-record DfltAccessPolicy
tunnel-group 192.102.223.7 type ipsec-l2l
tunnel-group 192.102.223.7 general-attributes
default-group-policy VPN_Policy
tunnel-group 192.102.223.7 ipsec-attributes
ikev2 remote-authentication pre-shared-key *****
ikev2 local-authentication pre-shared-key *****
!
class-map inspection_default
match default-inspection-traffic
!
!
p

Relavent Home 5506-X Config:

Code Select Expand

interface GigabitEthernet1/1
nameif Outside
security-level 0
ip address xxx.xxx.xxx.xxx 255.255.255.0
!
interface GigabitEthernet1/2
nameif inside
security-level 100
ip address xx.xx.xx.xx 255.255.255.0
!
same-security-traffic permit inter-interface
object network Home-Networks
subnet 10.0.0.0 255.0.0.0
object network Branch-WLAN-External
subnet 192.168.10.0 255.255.255.0
object network Branch-LAN-External
subnet 192.168.11.0 255.255.255.0
object-group network Branch-Networks
network-object object Branch-LAN-External
network-object object Branch-WLAN-External
access-list inside_access_in extended permit ip any object-group Branch-Networks
access-list inside_access_in extended permit ip object-group Branch-Networks any
access-list Branch_ACL extended permit ip object Home-Networks object-group Branch-Networks
access-list Branch_ACL extended permit ip object-group Branch-Networks object Home-Networks
nat (inside,Outside) source static any any destination static Branch-Networks Branch-Networks no-proxy-arp route-lookup
nat (inside,Outside) source static Home-Networks Home-Networks destination static Branch-Networks Branch-Networks no-proxy-arp route-lookup
access-group inside_access_in in interface inside
route Outside 0.0.0.0 0.0.0.0 xxx.xxx.xxx.xxx 1
route inside 10.0.0.0 255.0.0.0 xx.xx.xx.xx 1
crypto ipsec ikev2 ipsec-proposal Site2Site
protocol esp encryption aes-256
protocol esp integrity sha-1
crypto ipsec ikev2 ipsec-proposal DES
protocol esp encryption des
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal 3DES
protocol esp encryption 3des
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal AES
protocol esp encryption aes
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal AES192
protocol esp encryption aes-192
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal AES256
protocol esp encryption aes-256
protocol esp integrity sha-1 md5
crypto ipsec security-association pmtu-aging infinite
crypto dynamic-map SYSTEM_DEFAULT_CRYPTO_MAP 65535 set ikev1 transform-set ESP-AES-128-SHA ESP-AES-128-MD5 ESP-AES-192-SHA ESP-AES-192-MD5 ESP-AES-256-SHA ESP-AES-256-MD5 ESP-3DES-SHA ESP-3DES-MD5 ESP-DES-SHA ESP-DES-MD5
crypto dynamic-map SYSTEM_DEFAULT_CRYPTO_MAP 65535 set ikev2 ipsec-proposal AES256 AES192 AES 3DES DES
crypto dynamic-map Branch_Map 4 match address Branch_ACL
crypto dynamic-map Branch_Map 4 set pfs group5
crypto dynamic-map Branch_Map 4 set ikev2 ipsec-proposal Site2Site
crypto dynamic-map Branch_Map 4 set reverse-route
crypto map Outside_map 65534 ipsec-isakmp dynamic Branch_Map
crypto map Outside_map 65535 ipsec-isakmp dynamic SYSTEM_DEFAULT_CRYPTO_MAP
crypto map Outside_map interface Outside

crypto ikev2 policy 1
encryption aes-256
integrity sha
group 5 2
prf sha
lifetime seconds 86400
crypto ikev2 policy 2
encryption aes-256
integrity sha512
group 24
prf sha512
lifetime seconds 86400
crypto ikev2 policy 3
encryption aes-256
integrity sha
group 5 2
prf sha
lifetime seconds 86400
crypto ikev2 policy 10
encryption aes-192
integrity sha
group 5 2
prf sha
lifetime seconds 86400
crypto ikev2 policy 20
encryption aes
integrity sha
group 5 2
prf sha
lifetime seconds 86400
crypto ikev2 policy 30
encryption 3des
integrity sha
group 5 2
prf sha
lifetime seconds 86400
crypto ikev2 policy 40
encryption des
integrity sha
group 5 2
prf sha
lifetime seconds 86400
crypto ikev2 enable Outside client-services port 443
crypto ikev2 remote-access trustpoint ASDM_TrustPoint0


group-policy Branch_Policy internal
group-policy Branch_Policy attributes
vpn-tunnel-protocol ikev2
tunnel-group Branch_Map type ipsec-l2l
tunnel-group Branch_Map general-attributes
default-group-policy Branch_Policy
tunnel-group Branch_Map ipsec-attributes
ikev2 remote-authentication pre-shared-key *****
ikev2 local-authentication pre-shared-key *****
!


If someone could point out my idiocy to me, I'd appreciate it. 

[routerdork]

You have two NAT rules that match the traffic. I'm not sure why you would need the first one since it does the same as the second plus more. But in my experience it's needed to have the more specific NAT's before the "any" NAT's. I would either move up the second entry to the first position or get rid of the first one.

Also another thing that comes in handy for me is packet tracer. It will be very good at telling you what/where the packets are going. Run this from Home "packet-tracer input inside tcp 10.0.0.100 80 192.168.11.100 80 detailed".

[heath]

The extra NAT statement is probably from me throwing things at it to see if it changed anything.

Output of the packet-tracer:

Code Select Expand

PPhase: 1
Type: ACCESS-LIST
Subtype:
Result: ALLOW
Config:
Implicit Rule
Additional Information:
Forward Flow based lookup yields rule:
in  id=0x2aaac1810d30, priority=1, domain=permit, deny=false
        hits=52172, user_data=0x0, cs_id=0x0, l3_type=0x8
        src mac=0000.0000.0000, mask=0000.0000.0000
        dst mac=0000.0000.0000, mask=0100.0000.0000
        input_ifc=inside, output_ifc=any

Phase: 2
Type: ROUTE-LOOKUP
Subtype: Resolve Egress Interface
Result: ALLOW
Config:
Additional Information:
found next-hop xxx.xxx.xxx.xxx using egress ifc  Outside

Phase: 3
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
nat (inside,Outside) source static any any destination static Branch-Networks Branch-Networks no-proxy-arp route-lookup
Additional Information:
NAT divert to egress interface Outside
Untranslate 192.168.11.100/80 to 192.168.11.100/80

Phase: 4
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group inside_access_in in interface inside
access-list inside_access_in extended permit ip any object-group Branch-Networkss
object-group network Branch-Networkss
network-object object Branch_LAN-External
network-object object Branch_WLAN-External
Additional Information:
Forward Flow based lookup yields rule:
in  id=0x2aaac2b29a80, priority=13, domain=permit, deny=false
        hits=0, user_data=0x2aaabb980900, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
        dst ip/id=192.168.11.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0
        input_ifc=inside, output_ifc=any

Phase: 5
Type: NAT
Subtype:
Result: ALLOW
Config:
nat (inside,Outside) source static any any destination static Branch-Networkss Branch-Networkss no-proxy-arp route-lookup
Additional Information:
Static translate 10.0.0.100/80 to 10.0.0.100/80
Forward Flow based lookup yields rule:
in  id=0x2aaac21dc700, priority=6, domain=nat, deny=false
        hits=2, user_data=0x2aaac2b1e620, cs_id=0x0, flags=0x0, protocol=0
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
        dst ip/id=192.168.11.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0
        input_ifc=inside, output_ifc=Outside

Phase: 6
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in  id=0x2aaac1323050, priority=1, domain=nat-per-session, deny=true
        hits=2368, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
        dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
        input_ifc=any, output_ifc=any

Phase: 7
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in  id=0x2aaac1819ed0, priority=0, domain=inspect-ip-options, deny=true
        hits=1683, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
        dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
        input_ifc=inside, output_ifc=any

Phase: 8
Type: VPN
Subtype: encrypt
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
out id=0x2aaac2b2a460, priority=69, domain=encrypt, deny=false
        hits=2, user_data=0x164994, cs_id=0x2aaac23c53f0, reverse, flags=0x0, protocol=0
        src ip/id=10.0.0.0, mask=255.0.0.0, port=0, tag=any
        dst ip/id=192.168.11.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0
        input_ifc=any, output_ifc=Outside

Phase: 9
Type: NAT
Subtype: rpf-check
Result: ALLOW
Config:
nat (inside,Outside) source static any any destination static Branch-Networkss Branch-Networkss no-proxy-arp route-lookup
Additional Information:
Forward Flow based lookup yields rule:
out id=0x2aaac21db6e0, priority=6, domain=nat-reverse, deny=false
        hits=3, user_data=0x2aaac2b8ac70, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
        dst ip/id=192.168.11.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0
        input_ifc=inside, output_ifc=Outside

Phase: 10
Type: VPN
Subtype: ipsec-tunnel-flow
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in  id=0x2aaac23c4f70, priority=69, domain=ipsec-tunnel-flow, deny=false
        hits=973, user_data=0x16acec, cs_id=0x2aaac23c53f0, reverse, flags=0x0, protocol=0
        src ip/id=192.168.11.0, mask=255.255.255.0, port=0, tag=any
        dst ip/id=10.0.0.0, mask=255.0.0.0, port=0, tag=any, dscp=0x0
        input_ifc=Outside, output_ifc=any

Phase: 11
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in  id=0x2aaac1323050, priority=1, domain=nat-per-session, deny=true
        hits=2370, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
        dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
        input_ifc=any, output_ifc=any

Phase: 12
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in  id=0x2aaac17bee60, priority=0, domain=inspect-ip-options, deny=true
        hits=2511, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
        dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
        input_ifc=Outside, output_ifc=any

Phase: 13
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Config:
Additional Information:
New flow created with id 3245, packet dispatched to next module
Module information for forward flow ...
snp_fp_tracer_drop
snp_fp_inspect_ip_options
snp_fp_tcp_normalizer
snp_fp_translate
snp_fp_adjacency
snp_fp_encrypt
snp_fp_fragment
snp_ifc_stat

Module information for reverse flow ...
snp_fp_tracer_drop
snp_fp_inspect_ip_options
snp_fp_ipsec_tunnel_flow
snp_fp_translate
snp_fp_tcp_normalizer
snp_fp_adjacency
snp_fp_fragment
snp_ifc_stat

Result:
input-interface: inside
input-status: up
input-line-status: up
output-interface: Outside
output-status: up
output-line-status: up
Action: allow


[Reggle]

As you described it, you sent a packet from branch to headend through the VPN expecting a response. However, as I read your headend config, you don't have any firewall policy on Outside, meaning it will drop all traffic on the outside interface. If a VPN encrypted packet is received, it's probably being decrypted, doesn't match a policy, and it dropped. Hence no reply.
Either make a policy for it, or try a connection (ping?) the other way around and see if the numbers on the counters change places as well.

[heath]

Figured out the problem.  It was a routing issue.

[deanwebb]

Figured out the problem.  It was a routing issue.

I am pleased.

[heath]

   Love it!

[NetworkGroover]

I am pleased.

It's not the firewall - for once.

[deanwebb]

I am pleased.

It's not the firewall - for once.

It's NEVER the firewall. EVER.

[NetworkGroover]

 

[Dieselboy]

I am pleased.

It's not the firewall - for once.

It's NEVER the firewall. EVER.

If you use Cisco ZBFW on IOS then it might be the firewall that one time.

[Dieselboy]

As you described it, you sent a packet from branch to headend through the VPN expecting a response. However, as I read your headend config, you don't have any firewall policy on Outside, meaning it will drop all traffic on the outside interface. If a VPN encrypted packet is received, it's probably being decrypted, doesn't match a policy, and it dropped. Hence no reply.
Either make a policy for it, or try a connection (ping?) the other way around and see if the numbers on the counters change places as well.

I've never seen or tested this. Would the ASA not route traffic and prevent traffic from the inside to the outside zone and prevent TCP connections establishing with this set up? I mentioned this because when VPN is configured on the ASA it seems to put it secret allow rules from outside to a hidden "self" zone to allow the VPN to work without configuring specific allow rules on the access list.

[DanC]

As you described it, you sent a packet from branch to headend through the VPN expecting a response. However, as I read your headend config, you don't have any firewall policy on Outside, meaning it will drop all traffic on the outside interface. If a VPN encrypted packet is received, it's probably being decrypted, doesn't match a policy, and it dropped. Hence no reply.
Either make a policy for it, or try a connection (ping?) the other way around and see if the numbers on the counters change places as well.

I've never seen or tested this. Would the ASA not route traffic and prevent traffic from the inside to the outside zone and prevent TCP connections establishing with this set up? I mentioned this because when VPN is configured on the ASA it seems to put it secret allow rules from outside to a hidden "self" zone to allow the VPN to work without configuring specific allow rules on the access list.

You're right DieselBoy, you don't need specific ACE's to allow ISAKMP and ESP, that's allowed by default when you enable the cryptomap and IKE on the interface. I seem to remember though that the ASA won't pass any traffic from a lower sec zone to higher without an ACL so not sure whether this would apply here or not. By default the ASA bypasses ACL's for VPN traffic so I guess it's just down to the order it processes the logic.

[deanwebb]

If you use Cisco ZBFW on IOS then it might be the firewall that one time.

ZBFW on IOS is not a firewall. It's a stopgap measure, due to a business being too impoverished or too stingy to lay out money for a proper firewall.

[icecream-guy]

If you use Cisco ZBFW on IOS then it might be the firewall that one time.

ZBFW on IOS is not a firewall. It's a stopgap measure, due to a business being too impoverished or too stingy to lay out money for a proper firewall.

ZBFW is more of a firewall than CBAC, at least it has "fire wall" in the acronym