How do apps send email?

Started by Dieselboy, June 30, 2017, 07:52:30 AM

Previous topic - Next topic

Dieselboy

How do web applications send email? I mean, we know how email works but when you really think about it it's email and there's lots of concerns involved when you get down to it. Things like:

- do I need a whole email server, with inboxes etc
- what about mail filtering for nasty emails like malware
- outbound email only or both inbound and outbound
- quota for a mailbox
- MX records
- email domain

and there's probably more to list.

Is there cloud application email service providers or are companies running full enterprise-like email systems for their cloud apps? OR am I missing something?

icecream-guy

Mail of SMTP is a small operating system with commands and variables one can use to send mail

haven't you ever telnetted to a mail server on port 25 and typed in EHLO?
use the ? Should provide some list of commands,

my guess would be that the application telnet to a corp mail server on port 25 uses permitted commands and set variables based on to: from: subject: and other mail headers, and then the mail server takes care of the routing.
:professorcat:

My Moral Fibers have been cut.

deanwebb

If you want the emails from the apps to arrive in other peoples' inboxes, you will need it to participate in the general email system. This will mean setting it up as an account on the company mail server. If the company does not yet *have* a mail server, then everyone is either using personal email (bad) or shouting at each other down the hall (also bad). If the mail server is in the cloud or on premises, the work needed to set up an email account for the app is pretty straightforward.

If one is setting up a new email server, make sure that it is properly hardened and spamproof. You will want to require authentication in order to send email, IE ***NO OPEN RELAYS*** and I cannot stress that enough. Having an open relay is a good way to get blacklisted and then used as a spam spewer, or vice versa, and either way it's a very sticky end. You will also want to have all the mail go through a spamfilter like CloudFlare or Spamhaus. Set up reverse DNS and all the other security measures recommended by antispam companies and treat those recommendations as requirements.

Typically, as long as port TCP 25 is open between the web app and the email server, then the necessary connectivity exists to allow the email connection. Routing may be needed to get things from A to B properly, but that's the basic setup. The app itself must also have been coded properly to speak with the mail server. For example, if the app does only SMTP and the server does only ESMTP, then the app needs to step up its game.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

Dieselboy

#3
Quote from: ristau5741 on June 30, 2017, 07:59:28 AM
Mail of SMTP is a small operating system with commands and variables one can use to send mail

haven't you ever telnetted to a mail server on port 25 and typed in EHLO?
use the ? Should provide some list of commands,

my guess would be that the application telnet to a corp mail server on port 25 uses permitted commands and set variables based on to: from: subject: and other mail headers, and then the mail server takes care of the routing.
I didn't mean that  :mrgreen:

I didn't elaborate enough. Lets say for example, ACME company has a cloud application residing in a hosted environment. That cloud application can send email notifications. How would that be achieved in terms of (high level) design? You could possibly use a open relay for outbound email. But how reliable would it be and would it be allowed for company use? I had a quick google but the results seems a 'paid for' relay is targeted at people who send spam.

What about if that cloud application received email, that would then require email boxes and a database or a full blown email system contained within the cloud application. Is that typically whats done? What happens in terms of spam filtering and malware? Seems like a lot of work!

deanwebb

YOU DO NOT WANT AN OPEN RELAY

:kiwf:

The cloud application would need a mailbox on your local mailserver and then be permitted to connect to it via your spam blocker.

That, or set up a VPN between the app and the local network so it can connect directly - with authentication - to your mail server.

YOU DO NOT WANT AN OPEN RELAY. EVER.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

Dieselboy

 :XD: :XD:

I was looking for the opposite of 'open relay'. But their charging model charges per email address you send to. They described that if you send one email to 50 email addresses, then it's 50 "relays" and the relays are sold in blocks.

I just wondered how this sort of thing is typically done, because email is basic but to implement it properly it's pretty involved as there's all sorts of considerations that I've not mentioned too. I think if it's just sending outbound then a paid for relay might be a solution. I think the trick would be finding a company that could provide that as a service reliably. OR set up your own relay that's secure and only for yourselves. You would need the reverse DNS and a email domain I think.

Receiving email seems like a complete different story / world of pain!

deanwebb

Is this a personal application or for a business? If it's a personal one, maybe we can work something with this domain's SMTP server, shouldn't be too hard.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

Dieselboy

It's neither, I was pondering on ideas and I got stuck on this one.

How does emails work for you Dean? Do you have the ability to receive as well or is it only send?

I am keen to know if there are scalable solutions as well. Maintaining db's I know is tricky too when it comes to high availability.

deanwebb

Most web hosts will have a multi-tenant email server that they will use for customers. With each of my domains, Dreamhost set me up with an MX record for mail.domain.com and that's where all the SMTP communication comes and goes.

In the forums here, I set up a username, password, and server combination in settings so that the forums can send emails to me when needed.

Dreamhost takes care of all the HA, backup, stuff like that. I have a spam filter front end set up that will keep the inbox clean.

For a single-tenant solution, you're looking either at running Sendmail on a Linux host or going with the old warhorse, Microsoft Exchange. I'm partial to Exchange because I've supported it in the past and know that it has some very rich features and is very reliable. It also costs money, but that's why one gets a business to pay for it...
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

wintermute000

#9
Surely there are plenty of cloud providers as well (like every hosting provider as per Dean). Why run email as IaaS and take on all the burden when you can get it as PaaS/SaaS unless its a core component of your product.

I used to run my own email on my own domain - postfix+ dovecot + spamassassin + roundcube for webmail frontend on mysql in a VPS. Proper certs for TLS, self hosted domain and MX record with SPF, subscription to blacklists, passes all the anti-spam checks (can't even remember all the tweaks). Even managed to nail most of the spam after working out the ungodly chain between postfix and spamassassin and dovecot and SQL. I spent so much time neck deep in linux tutorials to get it all humming and periodically updated / adding new stuff.

Gave that up last year and just pay for it now as a service - I conceded I was not interested in learning the details of email servers, the SQL backend, or how to get them running redundantly/clustered. Way too much effort to keep up with what for me is non-core knowledge. Maybe different for you as you're a masochist ;)

If its for your company's product though, totally recommend going PaaS/SaaS and outsourcing the problem, including the anti-spam/AV and domain hosting. That's one less piece of infra redundancy you have to worry about.

Dieselboy

Nice, so it's as I thought then - do it yourself properly = a ton of work. 8)

icecream-guy

#11
Quote from: Dieselboy on July 02, 2017, 04:17:58 AM
Nice, so it's as I thought then - do it yourself properly = a ton of work. 8)

+ fun if you run Sendmail on linux.
++ fun if you also run bind and manage your own dns.
:professorcat:

My Moral Fibers have been cut.

deanwebb

Quote from: Dieselboy on July 02, 2017, 04:17:58 AM
Nice, so it's as I thought then - do it yourself properly = a ton of work. 8)


As a former email admin, you are correct, sir.

But, the services provided with nearly every web hosting arrangement are usually more than enough for a home lab or even home business.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.