Tuesday, 13 April 2010

IIS 7 Gotcha! 404.13 Not Found – Uploading Large Files with Integrated Pipeline!

Whilst uploading a large (70MB) file to an IIS 7 website I got a 404 error….which was odd, uploading a file in a postback shouldn’t give me that.  I know that file exists!

On further investigation it turns out it was actually a 404.13 error from the Request Filtering feature of the Integrated Pipeline (more info: http://bit.ly/dkZiYr). 

To fix this I needed to add some additional configuration to the <system.webServer> element on top of the <httpRuntime> modifications – not the subtle change of units!

   1: <?xml version="1.0" encoding="utf-8"?>
   2: <configuration>
   3:     <system.web>
   4:         <!-- maxRequestLength and requestLengthDiskThreshold is in Kilobytes-->
   5:         <httpRuntime maxRequestLength="204800"
   6:                      requestLengthDiskThreshold="204800" />
   7:     </system.web>
   8:     <system.webServer>
   9:         <security>
  10:             <!-- maxAllowedContentLength is in Bytes not Kilobytes -->
  11:             <requestFiltering>
  12:                 <requestLimits maxAllowedContentLength="204800000" />
  13:             </requestFiltering>
  14:         </security>
  15:     </system.webServer>
  16: </configuration>

Saturday, 10 April 2010

My Windows 7 Development VM Load Out

Well, it was bound to happen.  I rebuilt my PC and forgot to install my dev tools… oh well.  In an attempt to stop this from ever happening again, I’m creating a dev VM which will be backed up onto my NAS device.

The applications I’m installing for (ASP.Net Development) are:

Thursday, 8 April 2010

Secure Your Website, Put Bouncers at the Door (Part 3)

In part 1, I gave an overview of the similarities in securing a good night club and a website and in part 2 I gave an overview of what we, as developers, can do to secure our websites.  In this final part, I give details on how we should be doing the most important (and mundane) part of securing our websites:

1. Input Validation (Continued)

One important fact that many web developers fail to take on board, is that everything you receive from the client is a string; Query strings, post back values and cookies.  It may look like something else, but it’s actually a little crack in the armour of your website.
For example, a query string of  “?id=56” doesn’t mean that id is equal to the integer 56, only it’s string form.  It can just as easily be manipulated to read “?id=Bob+The+Builder” or the more vicious “id=’);drop table Users;”.   If you’ve ever used string concatenation in a SQL statement, your blood should have just run cold!
An excellent habit to get into is to convert every parameter you need as early as possible in the request and use that value from that point on, if the value can’t be converted then throw an exception to stop processing the request.  This is one of your websites bouncers, it’ll let through good values but prevent the troublemakers from ruining it for everyone.
This type of conversion is simple, but can be a little long winded to do in a best practice way.  The two most common ways I see of doing this in .Net is by using the Convert class or the target types Parse method.  These are nice quick one line commands to parse a string into an integer, such as:
   1: int id = Convert.ToInt32(Request.QueryString["id"]);
The above code will work fine as long as the string does actually represent that data type, otherwise you need some boiler plate code to correctly handle the ‘FormatException’ that gets thrown:
   1: int id=0;
   2: try
   3: {
   4:     id = Convert.ToInt32(Request.QueryString["id"]);
   5: }
   6: catch(FormatException fex)
   7: {
   8:     throw new NotSupportedException("The querystring parameter 'id' must be an integer value");
   9: }
Of course, you don’t need to wrap the FormatException but it will make debugging easier if there was a little more information than ‘something wasn’t an formatted correctly’.  These methods also have a performance hit associated with them as they throw exceptions on failure (that’s fine for this example – but in others, you may have default values that should be used) and each thrown exception causes the ASP.Net to prioritise handling that exception over running code….
So to avoid the exception there’s now (in .Net 2.0) TryParse:
   1: int id = 0;
   2: if (!Int32.TryParse(value, out id)) id = 0;
That’s better but still two lines of boiler plate code (and a slightly strange pattern) that will need to be parroted out for each parameter on every request and more boiler plate code means it’s less likely developers will stick to doing it.
Of course, this is exactly why the programming gods gave us the concept of Refactoring and Separation of Concerns, you can easily write a reusable wrapper method around the TryParse functionality to make it easier to code with.  Thanks to the good people of Microsoft it’s trivial to create an Extension method to provide the functionality to all strings:
   1: /// <summary>
   2: /// Gets the Int32 value stored in the string
   3: /// </summary>
   4: /// <param name="value">The string to parse</param>
   5: /// <param name="defaultValue">The default value if the the given value cannot be parsed</param>
   6: /// <returns>
   7: /// The integer represented by the string, otherwise the default value
   8: /// </returns>
   9: public static Int32 ToInt32(this string value, Int32 defaultValue)
  10: {
  11:    if (string.IsNullOrEmpty(value)) return defaultValue;
  12:    Int32? result = null;
  13:    Int32 temp;
  14:    if (Int32.TryParse(value, out temp)) result = temp;
  15:    return result ?? defaultValue;
  16: }
This makes our ‘id’ parsing example as easy as:
   1: int id = Request.QueryString["id"].ToInt32(0);
That’s a sweet, safe one line conversion that’s fewer keystrokes than the most basic Convert and Parse methods. 
To save everyone and their dog from creating versions of these wrappers, I’ve created a small class that provides localization aware type conversion methods.
Attachment: Type Converter Helper
Conversion Performance
I know that some developers feel particularly attached to their ‘Convert’ and ‘Parse’ methods so I’ve created a little performance comparison app and gathered some performance statistics:
Method 5000 Successful Parses (“12345”) 5000 Failed Conversions (“Not An Integer”)
Convert 6ms 52s
Parse 7ms 54s
TryParse 6ms 6ms
TryParse (Extension) 6ms 6ms
This shows that when everything runs as expected (the ‘Happy Path’) there’s not much difference, but  when invalid data is given there’s a 10000x performance decrease.  That should be enough to get even the most entrenched Convert/Parse only developer to change their ways!
The application I used to create these numbers is included in the helper library project.

Ektron: A Fix For Uploading Assets into a Folder Called Assets

An interesting bug came in today from a client that had been uploading content and images into their workarea for their upcoming release.  Most of their images weren’t being displayed on the site or in the workarea.

When the image url was navigated to directly, instead of an server error or file not found we saw this:

Error: Asset requested doesn't exist in the system

I beg to differ! I can see it right there on the file system!  After a bit of Googling, I found the same issue on the Ektron forums (http://bit.ly/90erho)  which shows this has been a problem since (at least) 2008!  It’s also a reasonable Google Hack to find other Ektron Deployments with the same issue (I see you Walmart!).

With a bit of digging I traced the source of the issue to the EkDavHttpHandlerFactory Handler that’s used for the url rewriting.  More specifically, this block of code:

   1: // Taken from Ektron.ASM.EkHttpDavHandler.EkDavHttpHandlerFactory.GetHandler using Reflector
   2:  
   3: if (((context.Request.PhysicalPath.ToLower().IndexOf(@"\assets\") >= 0) 
   4:     && (context.Request.PhysicalPath.ToLower().IndexOf(@"\thumb_") < 0)) 
   5:     && ((context.Request.PhysicalPath.ToLower().IndexOf(@"\orig_") < 0) 
   6:     && (context.Request.PhysicalPath.ToLower().IndexOf(@"_indexed\") < 0)))
   7:   {
   8:       return new AssetHttpHandler();
   9:   }

D’oh!  The images in question had been uploaded to a folder called assets within the library (not the ~/assets/ folder but ~/uploadedImages/…./assets/….) and the overzealous detection algorithm above was treating them as DMS items.

To fix this I’ve implemented a subclass of the handler which performs extra checking when the AssetHttpHandler IHttpHandler implementation is returned to ensure that it’s only blocking assets within the ~/assets/ folder.

   1: using System;
   2: using System.Web;
   3: using System.Reflection;
   4: using global::Ektron.ASM.FileHandler;
   5: using global::Ektron.Cms.UrlAliasing;
   6:  
   7:  
   8: namespace MartinOnDotNet.Ektron.Web
   9: {
  10:     /// <summary>
  11:     /// Correctly handler asset folders named /assets/ that aren't the ektron assets folder
  12:     /// </summary>
  13:     /// <remarks>Required assembly references
  14:     /// <list>
  15:     /// <item>Ektron.ASM.EkHttpDavHandler</item>
  16:     /// <item>Ektron.ASM.FileHandler</item>
  17:     /// <item>Ektron.Cms.URLAliasing</item>
  18:     /// </list>
  19:     /// </remarks>
  20:     [CLSCompliant(false)]
  21:     public class EkDavHttpHandlerFactory
  22:         : global::Ektron.ASM.EkHttpDavHandler.EkDavHttpHandlerFactory
  23:     {
  24:  
  25:         /// <summary>
  26:         /// Gets the handler.
  27:         /// </summary>
  28:         /// <param name="context">The context.</param>
  29:         /// <param name="requestType">Type of the request.</param>
  30:         /// <param name="url">The URL.</param>
  31:         /// <param name="pathTranslated">The path translated.</param>
  32:         /// <returns>The correct request handler</returns>
  33:         public override System.Web.IHttpHandler GetHandler(HttpContext context, string requestType, string url, string pathTranslated)
  34:         {
  35:             IHttpHandler handler = base.GetHandler(context, requestType, url, pathTranslated); // inherit other mapping logic
  36:             if (typeof(AssetHttpHandler).IsInstanceOfType(handler)) // override assets folder handling
  37:             {
  38:                 if (!context.Request.PhysicalPath.StartsWith(context.Server.MapPath("~/assets/"), StringComparison.OrdinalIgnoreCase))
  39:                 {
  40:                     handler = GetFallbackHandlerForServer(context);
  41:  
  42:                 }
  43:             }
  44:             return handler;
  45:         }
  46:  
  47:         /// <summary>
  48:         /// Gets the fallback handler for server.
  49:         /// </summary>
  50:         /// <param name="context">The context.</param>
  51:         /// <remarks>This algorithm was taken from the Ektron implementation using 
  52:         /// Reflector.  As it's reflection based, there's a chance that future releases
  53:         /// will break it.</remarks>
  54:         private static IHttpHandler GetFallbackHandlerForServer(HttpContext context)
  55:         {
  56:             if (ServerVersion(context) > 6)
  57:             {
  58:                 return new StaticFileHandler();
  59:             }
  60:             return new DefaultHttpHandler();
  61:         }
  62:  
  63:         private static int? _serverVersion;
  64:  
  65:         /// <summary>
  66:         /// Servers the version.
  67:         /// </summary>
  68:         /// <param name="current">The current.</param>
  69:         /// <returns>The server version using the ektron implemented algorithm</returns>
  70:         public static int ServerVersion(HttpContext current)
  71:         {
  72:             if (!_serverVersion.HasValue)
  73:             {
  74:                 Type t = typeof(global::Ektron.ASM.EkHttpDavHandler.EkDavHttpHandlerFactory);
  75:                 _serverVersion = t.InvokeMember(
  76:                     "GetServerVersion"
  77:                     , BindingFlags.InvokeMethod | BindingFlags.NonPublic | BindingFlags.Static
  78:                     , null
  79:                     , t
  80:                     , new object[] { current }
  81:                     , System.Globalization.CultureInfo.InvariantCulture) as int?;
  82:             }
  83:             return _serverVersion.GetValueOrDefault(6);
  84:  
  85:         }
  86:     }
  87: }

Where possible I’ve reused the functionality from the API (even using a nasty bit of reflection to extract the server version according to the Ektron API). 

To use this, copy the class above into a your websites \App_Code\CSCode\ folder and in the web.config replace all references to ‘Ektron.ASM.EkHttpDavHandler.EkDavHttpHandlerFactory’ with ‘MartinOnDotNet.Ektron.Web.EkDavHttpHandlerFactory’.

Tuesday, 6 April 2010

Secure Your Website, Put Bouncers at the Door (Part 2)

In part 1, I gave an overview of the similarities in securing a good night club and a website.   In this part I’ll give an overview of what we as developers can do to implement these measures:

4 and 5 Authentication and HTTPS

The implementation of these is very much driven by the business requirements of the site, if the site’s free to access and there’s no personal information they’re a non-issue.  There’s plenty of good documentation on implementing authentication on ASP.Net and if you require HTTPS then I can’t recommend Matt Sollars’ Automatic HTTPS switching module enough.

3. Robots.txt

This is the file that no website should be without!  If your website has an administration, registration,login or otherwise secure page that you don’t want people to find easily via a quick Google hack, here’s the place to stop it being indexed!   It’s so important it has an entire website dedicated to it, it’s a two minute job that could save masses of embarrassment (or data loss).  An example robots.txt could be:
   1: Sitemap: http://www.example.com/sitemap.xml
   2:  
   3: User-agent: *
   4: Disallow: /secure/
   5: Disallow: /admin/
   6: Disallow: /googlehackablepages/
   7: Disallow: /configs/

2. Input Sanitation

This is another no-brainer.  Don’t concatenate strings into SQL statements  - ever!  Use parameterised queries be they stored procedure or dynamic queries there’s simply no excuse.

For XSS injection this is a little trickier,   I believe in escaping/html encoding text coming out of a database rather than when it’s going in simply because you can never be 100% certain that someone hasn’t fiddled the data in the database.  Microsoft have released a good XSS encoding utility recently to handle this.

1. Input Validation

By far the most important web development task is input validation.  This has to happen on every request for every value that’s received from the client, this is your websites front door.  Unfortunately, it’s also pretty mundane so there’s not much out there on the best practise.  Lot’s on ASP.Net validation controls, which handle your business rule validation but not much on minimising the surface area of attack for your website.
As a developer you should have some pretty strict expectations about what will be contained in a variable and where it comes from.  For example: if you use a CMS, you should know that the id for the content that your displaying is referenced is the ‘id’ query string value and should be a positive integer.  Anything, outside of this is an erroneous value and should be rejected.
In this example there’s a implied checks that we need to test every time we read in that value:
  1. The ‘id’ value is always read from the query string (not a cookie or form variable)
  2. The ‘id’ is a whole number (integer)
  3. The ‘id’ is always greater than zero
(and that’s before we check to see if the ‘id’ actually matches any content!)
We can perform these tests easily enough by reading the value explicitly from the query string, converting it to an integer and testing that it’s greater than zero:
   1: string value = Request.QueryString["id"]; //Not Request["id"] as this will hunt for a matching key in querystring, form variables and cookies
   2: int id = 0;
   3: if (!Int32.TryParse(value, out id)) id = 0;
   4: if (id<=0) throw new NotSupportedException(string.Format(CultureInfo.InvariantCulture, "The querystring parameter 'id' must be a positive integer value, the value given was '{0}'",Request.QueryString["id"]));
But that’s a lot of code that needs to be written each time a variable is read from a request….in part 3, I’ll go into the ‘nitty-gritty’ input validation and take a look at the most common methods of parsing values and present much tidier code.

Monday, 5 April 2010

Educating Clients About The Way the Web Works

Craig’s pointed out an interesting discussion about managing client expectations (that kicked off from a very preachy design article) around the final display of a website across browsers.   This sort of discussion has been around for a while now and I think clients are starting to wise up to browser display issues.

Personally, I’m fairly indifferent about how most websites look provided they behave as I expect don’t actively get in my way. I expect that most users of websites feel pretty much the same way although web designers will probably disagree. Where these design issues are very obvious to end users, website implementation issues aren’t.  So the general level of understanding from clients it quite low, particularly when it comes to localized websites.

Multilingual and Multi-branded

For large corporate clients I’ve found that the client wants to represent their internal subdivisions and have similar general requirements:

  • A  global site with:
    • Micro sites for territories (country sites)  with English copy with native translations
    • Micro sites for brands
  • News pages for each of the above
  • News feeds for each of the above
  • Contact pages for each of the above
  • Search pages for each of the above
  • A single point of administration

Now as a web developer, I see this as being implemented as a single white-label codebase that has multiple brands/themes that represent different aspects of the same company maintained through a single admin screen. 

As a user (albeit with a bit of knowledge about the web), I’d expect it to be presented as:

  • A ‘client.com’ site for the global website with generic (targeted at Americans) overview information about the company/services that’s written in Americanised English – so describing Paris as “Paris, France” would be an acceptable in copy
  • A set ‘brand.client.com’ sites that again give generic information about that particular brand and with any appropriate TLDs for localized copy/information
  • A set of country sites (with appropriate TLDs) (client.co.uk, client.fr, client.de) that display localized content in the native language for that country.  As a Briton (born in Scotland, raised in England), it’s really irritating to see “color” and “Paris, France” on a .co.uk website.  It’s snubs like this that really put me off a company (localized isn’t the same as translated – there’s a difference between en-GB and en-US!)

What the Web Wants, it Should Get!

Putting my web developer hat on again, if these were the requirements and expectation delivering the website would be relatively straightforward, each territory/brand website is a white label brand the locale can be picked up from the TLD domain.

Google expects this, Ektron expects this (as would must CMS’s that support multiple sites) and users expect this.

But the client doesn’t want this.  The client wants:

  1. All the content under the global domain
  2. To re-use copy between territories/brands
  3. For localization to happen ‘auto-magically’ in the background
  4. For Google to ‘auto-magically’ find the localized content for each territory

On top of this there’s actually the un-spoken needs:

  • It should be reliable
  • It should be fast
  • The site should be intuitive
  • It shouldn’t put people of the brand
  • It should be delivered promptly

Now, there’s nothing in the clients “wants” list that can’t be delivered, but each item will impact at least one thing from the “needs” list and as we’re now coding around the conventions there’ll be an ongoing maintenance cost as well.  It’s important that the client understands this.

So the client will need to educated about the inherent complexities about their website, to make it appear simple and work simply for the future.  They need to stop thinking about the website as purely as a marketing tool, but as an adapter making their company accessible to web users via the website.

It wouldn’t hurt long-term SEO either.

Sunday, 4 April 2010

Secure Your Website, Put Bouncers at the Door (Part 1)

A good website is much like a nightclub, we want the general population to be able to use the facilities freely (or for a nominal fee!) but we don’t want the trouble makers that will spoil it for everyone.   Pushing the analogy further, nightclubs have a few different types of security in place:

Visibility Implementation Mitigates
High Profile Bouncers
Filter out trouble makers at point of entry
  Internal Security Eject people that become troublemakers
Low Profile Bars on Windows
Alarmed Exits
Prevent people sneaking in
  Metal Detectors
Security Cameras
Passively check for people about to cause trouble

As web developers we have a very similar set of tools that we can use to secure our website:

Visibility Implementation Mitigates
High Profile Input Validation
(Metal Detectors)
Filters out bad information early
  Authentication
(Bouncer)
On allow members on to the site
  HTTPS
(Bars on Windows)
Prevent people monitoring user traffic
Low Profile Input Sanitation
(Security Cameras)
Filter out bad requests
  Robots.txt
(Bars on Windows)
Hide sensitive pages from popular web spiders
  Network Segregation and Infrastructure
(Alarmed Exits )
Control access to sensitive information

There’s lots of information out there about configuring the High Profile security measures and Network Segregation and Infrastructure is best implemented by the network specialists (it doesn’t hurt to understand the basics) and these things are generally well implemented.

As web developers, we have to worry about the things that happen inside the website, making sure that the front entrance is the only way in and limiting the damage if someone does get in.  So our implementation priorities for any website have to be:

  1. Input Validation – every piece of information that comes from the client is suspect and should be validated, this includes query string, form fields (even hidden ones!) and cookie values
  2. Input Sanitation  - ensure the any inject SQL or XSS script is neutralised
  3. Robots.txt – it may not be a bullet proof defence if someone’s targeting your site, but at least you can minimise drive-by hacking through Google!
  4. Authentication – allow only authorised users through
  5. HTTPS – secure the traffic from client to server

In Part 2, I’ll go over some of the best practises that we can follow, as developers, to make our website secure.