Codex Securitatis

The problem:
The user is expected to enter his email address in a form field.

The solution(s):

1. Pass it through a simple check to see if it contains a @ sign.
2. Pass it through a regexp checking for “name@domain.com” format.
3. Pass it through a really complex regexp checking for full RFC 2822 correctness.
4. Pass it through a really working really complex regexp checking for really full RFC 2822 correctness.
5. Pass it through a really working really complex regexp checking for really full RFC 2822 correctness, and then you check if the domain in the domain part exists.
6. Send a unique random token (in a link back to your site) to the email.

Email addresses can take really really complex forms, and still be RFC-valid. They can be RFC-valid, but the particular email provider may like only a subset of what the RFC requires (for example, for the local part - think “username” - of the email gmail allows only alphanumerics and dots, 6-30 characters). The domain name may not exist. It may exist, but not have an MX record (i.e. it will not accept mails). The domain may accept mails, but the local part may point to an non-existing account.

So what do we do? We use the last of the above options: we send a mail, and wait for proof that it’s readable by the user. But why don’t we do a validity check first? At least some sanity check? Okay, but keep it as a warning only: tell the user if the address doesn’t “look” valid, but don’t expect that your Really Correct Check ™ is actually correct, so don’t deny the form submission. Unless you’re okay with angry users / customers of course.

What if you want an email address that does not belong to the user (think e-cards). Well, then you sigh, pick one of the checking methods above except the last one, use it as a warning only (not as a fail test), and just send the mail and hope for the best.

Obviously, in both cases (whether the email is owned by the user or not), there is the problem that we’ll send a mail to a given email address without further questions. As we saw, we can’t really ask any meaningful question that will answer if the email is “valid”, besides sending an email, so we have a problem with spambots here. The solution is well known - deny access to the bots by protecting the form with a good strong CAPTCHA.

(Well, yes, it is, but it’s not correct to think so. Here’s why:)

What I will write about is applicable to all kinds of string injection (SQL injection, XSS, header injection, command line injection, etc.) vulnerabilities. Speaking in generalities (in order to encompass them all) will make the exposition much harder to read, so I will instead talk solely about SQL injection. The parallels with other “string injectables” rather than SQL queries (HTML pages, HTTP headers, etc.) are more or less straightforward.

An SQL injection attack happens (mostly) when malicious input breaks out of “value” context and leaks into the SQL “syntax” context. The most common example is when a quote is inserted and everything after the quote is interpreted as SQL statements. The offered remedy is to escape the values (among other things) so that all input will remain in the “value” context.

Here’s what the manual has to say (substitute “escape” for their “quote”) (emphasis mine).

Quote each non numeric user supplied value that is passed to the database with the database-specific string escape function.

Even ignoring that “non numeric”, the troubles start now:

What is “user supplied”? $_GET, $_POST, $_COOKIE, $_REQUEST are clearly user supplied. $_SERVER? Partly so. Database? It depends. (If you answered “no” to this one, go read about Second order SQL injection). Config vars? Nah, they’re fully in our control (hmmm)

First, let’s notice the lesser evil: time. After some of it, chances are that pieces of your code will get rewritten, patched or copy-pasted elsewhere. The assumption about the “user supplied”-ness of these values may change, and the risk of forgetting this assumption is great. (For an artificial example, imagine changing how a “default languge” setting could be redesigned - instead of being a site-wide config variable, it is now a per-user variable and kept in the database. Suddenly it becomes quite user-controllable)

The true “evil” is that we think about security, while we should have been thinking about the correctness of the code (”correct” in the sense of bug-free). Consider this: security holes are a subset of software bugs. If a piece of code is “bug free” (haha) it is also “secure”. And here comes the title: escaping is not a security measure! Escaping is a way to ensure that what we pass as “values” is indeed interpreted as a “value”.

Going back to the above questionary, imagine a config var containing a quote. It is not attacker-conrollable, so it’s not a security risk. It will still break any query it is inserted into withouth being escaped. All security holes are bugs, but not all bugs are security holes. All bugs should be fixed though.

The moral of all this long winded talk is this: don’t think of escaping as a security measure. Don’t think about “tainted” input and whether this $_SERVER index is user-controllable. Don’t “hide” assumptions in your code. Escape every value that goes in a query. This will fix not only the security holes, but also any possible data-related bug.

(This originally appeared here on 17 September 2007.)
I am pleased to announce that WASC has published a paper I wrote for the security articles project. The project is very nice, because all articles must pass through a critique and voting process by a peer group of security professionals (and I am proud to be among them when not wearing the hat of an author).

The Unexpected SQL Injection
(When Escaping Is Not Enough)

Abstract:
We will look at several scenarios under which SQL injection may occur, even though mysql_real_escape_string() has been used. There are two major steps at writing SQL injection resistant code: correct validation and escaping of input and proper use of the SQL syntax. Failure to comply with any of them may lead to compromise. Many of the specific issues are already known, but no single document mentions them all.
Although the examples are built on PHP/MySQL, the same principles apply to ASP/MSSQL and other combinations of languages and databases.

Full text: [HTML] [TXT] [ZIP (examples)]

(This originally appeared here on 31 October 2007.)

Much has been said about this brainfart of a feature, and attempts at reverting its behaviour are common for all php coders. Old versions of the php manual were giving this function in their “Best Practice” example:

1
2
3
4
5
6
7
8
9
10
11
12
13

// Quote variable to make safe
function quote_smart($value)
{
   // Stripslashes
   if (get_magic_quotes_gpc()) {
       $value = stripslashes($value);
   }
   // Quote if not integer
   if (!is_numeric($value)) {
       $value = "'" . mysql_real_escape_string($value) . "'";
   }
   return $value;
}

… which has apparently became a bad meme in more than one way. I have already mentioned (as many others did as well way before me) the blunder of using is_numeric(), but a more alarming mistake is the way magic quotes are handled. It isn’t bad just because it’s buggy, but because it gives a seriously flawed idea.

It is also bad, as it is (was) given as an example to newbies, and even after it was replaced in the manual, one can still see the meme “in the wild” in snippets posted on sites and fora. What’s worse, the manual didn’t explain what was wrong in the code before it was replaced; as I see it, maybe they still have no idea what wrongness they have been teaching.

So, after you’ve had a paragraph of reading time to think about it, do you see it? The bug lies in the assumption that the data that is given to quote_smart() comes from $_GET, $_POST, $_COOKIES, $_REQUEST, etc. If you pass something else (a constant string, a value from a file or database, etc.) containing slashes (for a superficial example, take a smb-style path: “\\host\share\file.ext”) and you have magic_quotes on, the function will blindly run stripslashes, and thus damage the string (“\hostsharefile.ext”).

The function also doesn’t check for the magic_quotes_sybase setting, which completely changes the way magic quotes are handled.

The correct way of negating magic_quotes is of course globally, at script startup, with proper setting checks and while keeping in mind that those arrays may contain other arrays.

But enough about the bug, it is something that happens with certain inputs under certain setups, and it damages the data in a way that doesn’t really affect the security. So what’s the big deal, is it worth writing so long a rant about it? I think so, and the reason lies in that implied assumption above, about where the data comes from. What it basically says is that input comes only from one of the input superglobal arrays, which is wrong. Input comes from all kinds of sources and you never know which of them may be under the control of an attacker.

So data coming from other places, take the database for example, gets labelled as “secure” in the mind of the coder, and he readily inserts it in a dynamic SQL query and thus second-order SQL injections are born. Such are the curses of bad memes, caveat coder.

Hello, world! I decided to split the security content from my other blog here. I promise to keep the content in a more up-to-date timeframe than peasant latin.

Here are the security-related articles I’m too lazy to move:

CORE GRASP (potential) pitfalls
Moved: The Unexpected SQL Injection
Moved: The Curse of Magic Quotes