Testing and configuring web apps but the browser wont let me

[Dieselboy]

Using Windows 10 (this time).
Using Browsers:
Chrome
Firefox
Internet Explorer
Edge

Issues:
1. trying to access the HTTP version of the page and the browser switches back to HTTPS when you press return or "go".
2. Closing all the tabs, opening the browser again, pasting in or typing out the "http' version of the URL and pressing "go" gives same issue as 1.
3. Using Firefox to access a HTTPS website to verify the SSL cert is loaded and configured correctly results in invalid cert warning in the browser. Even though I'm 90% sure the site is using a valid SSL cert
4. DNS to IP address mapping was updated, browser ignores it

Steps:
1. Azure VM goes down over night, next day comes up with new public IP (this is dev)
2. I update the dns record
3. Clear local dns cache, can now ping the DNS name and resolve the correct IP
4. try to load a new browser tab, page not loading
5. try a mix of closing browser, opening new tabs, or new browsers etc etc etc - will be resolved eventually (making coffee at this point helps).

ps. chrome has a clear dns cache button in the settings of the browser.

Fixing issue 3 was easy but not obvious. I had visited the site when it was using a self signed cert and added an exception to firefox. This appears to use this cert when you visit the site again, even though the site has a valid and different cert. I deleted the exception - problem gone. Site shows current and correct SSL cert in use. 

I'm stuck with issue 1 and 2.  The only browser I have found to work is internet explorer. However I needed to follow these steps:
1. open IE
2. paste in the url http://192.168.x.x
3. page does not load because the browser changed it to https://192.168.x.x
4. edit the url and change https to http and then press go
5. page loads

This has been progressively getting worse and worse as the browsers are trying fancy things to make simple end-users stay secure. I get that. But is there a web browser for engineers that just want to do their work on time?

[icecream-guy]

These days nothing should be running on http TCP/80,  TCP/8080. Any legit web services should be running on https TCP/443, TCP TCP/8443.

have you tried to force to port 80?  http://192.168.x.x:80

[deanwebb]

The .htaccess file on the server can force an HTTPS connection. I think I do that for this site. Basically, the only way to get HTTP when .htaccess forces HTTPS is to edit the .htaccess file. Forcing the port to 80 will get the server to force it right back to 443.

[Otanx]

But is there a web browser for engineers that just want to do their work on time?

curl? Ok, just kidding. I find that using private browsing helps for testing. It does not cache stuff (depending on the browser it will use the regular cache). As for the http to https problem. Do you know if the site has ever had, or has, HSTS configured? That is the first thing that came to my mind. HSTS lets a website tell a browser that it should only use https to connect in the future by setting a http header. Typical timeout for this is a year. So once you connect to it with https once it will not let you use http anymore.

-Otanx

[Dieselboy]

I get you, but this is for my normal day to day work. I'm not making a website. IE, this is not a legit website, so it should work how it's configured
I didnt try and force port 80 but a quick test looks like that works (at this time!).

For the SSL cert issue with firefox, this task specifically was a proxy for an API. My task that time was to hide it behind an SSL proxy. The cert was valid as it was issued by lets encrypt but firefox was showing the site as invalid due to the caching of the old cert, even though the site was no longer using it. I thought this was weird...

For the http / https issue, I was trying to access the web UI on a IOS-XE switch. HTTPS is not working and when I do a "no ip http secure-server" and then "ip http secure-server" it logs this:

Code Select Expand

Failed to generate persistent self-signed certificate.
    Secure server will use temporary self-signed certificate.

Although the web UI wont load, probably due to the cert error above. I wanted to do this for a test, to confirm I can reach the site during a failover. At one point, I just couldnt reach the UI via http even though it is available... I eventually installed an apache web server on the centos 7 install I done last week and configured the 2nd nic for the IP I needed, and shut down the 1st nic within the OS. Basically, a lot more work than just accessing the switch.

Curl worked fine! I'll try the private browsing next time, don't think I tried it this time as I had problems with that also at one point.

I used a web browser a long time ago, to connect to an IP and port to confirm / check SSL cert being offered on the connection. I tried this with postgresql port recently and the browser blocked that also. I understand why, but I think there should be a "Firefox for Engineers!" or "Chrome for Engineers!". If someone makes this, the "for Engineers!" part needs to be in italic

[DesertFox]

Have you considered using something like Burp / Zap and automatically change https to http? Should be possible, but I am not good with those (probably not a bad idea to have a deeper look myself).

[Dieselboy]

I am fully using Traefik proxy for a ton of things. So simple and easy to set up and I can add extra routes / proxy configs by telling traefik to watch a folder for changing configuration files. So, I keep each proxy config in it's own config file. Like Conflfuence, JIRA, SSO etc and can add new entries by adding a new file. No need to restart any services or container. Traefik uses Lets encrypt for SSL certs. I have configured traefik to accept HTTP but to redirect to HTTPS. I have a separate one internally that I use to proxy towards the backend servers. For example, Bitbucket listens on tcp:7990. In the past, we would use some weird nginx config or I think a bit better- using firewalld to NAT incoming http to http:7990 on the server. But using traefik means I dont need to configure SSL on each of the servers and applications. Just simply spin them up and then use HTTPS in a sort of microservice way. This also gets around the hurdle or applying SSL certs. For example, atlassian's apps (jira / bitbucket etc) use JAVA server. So to apply the SSL cert you have to do a fair few things and if you miss a step, you start the server and wait for error logs. It's a bit annoying, although presently my JIRA and Confluence is running this way. But going forward, I'll un-do the config for that, and enable the default HTTP once again. Then the proxy will do the HTTPS towards the client.

Not heard of burp / zap. Will look it up - thanks for the name dropping, I always appreciate that sort of thing. 👍

[Otanx]

Now I have to go read about Traefik. Everyone should learn something new every day. Every day you don't you fall further behind.

-Otanx

[Dieselboy]

Now I have to go read about Traefik. Everyone should learn something new every day. Every day you don't you fall further behind.

-Otanx

This should get you started https://www.networking-forums.com/everything-else-in-the-data-center/nginx-replacement-traefik-reverse-proxy-basic-setup-guide/

[Otanx]

Thanks, looks pretty cool. Went down the rabbit hole. Looks pretty easy to setup. Next time I end up on a web project I will have to look at this as an option.

-Otanx