Monday, 1 November 2010

SQL: Cleaning Up After the Database Tuning Engine Advisor

I’ve recently had to do a quick bit of DB performance analysis work on a 3rd Party database to see if I can improve performance(without being too invasive).  The quickest way to do this, is using the Database Tuning Engine Advisor tool that comes with SQL 2008 (the best way is to have loads of experience and a tonne of time to properly analyse the database).

Part of my analysis procedure was to make a script of queries that I could run with and without the DTA recommended indexes to evaluate any real world improvement.  This meant I needed to be able to roll back any indexes created by DTA.  Unfortunately, DTA gives you and install script for the new indexes but not a removal script.  I needed to write something myself.

Fortunately, the DTA created indexes and statistics are all prefixed with _dta_ so it’s relatively easy to write a query to remove the automatically created scripts:

First to drop the generated statistics:

   1: DECLARE @dtaStats AS TABLE(Id INT IDENTITY(1,1), StatName VARCHAR(255), TableName VARCHAR(255))
   2: DECLARE @currentId AS INT
   3: DECLARE @lastId AS INT
   4:  
   5: DECLARE @statName VARCHAR(255)
   6: DECLARE @tableName VARCHAR(255)
   7:  
   8: INSERT INTO @dtaStats(StatName,TableName)
   9:     SELECT name, OBJECT_NAME(OBJECT_ID) AS TABLENAME
  10:     FROM sys.stats
  11:     WHERE name like '_dta_stat_%'
  12:  
  13: SET @currentId=NULL
  14: SET @lastId=0
  15:  
  16: SELECT @currentId=MIN(Id)
  17: FROM @dtaStats 
  18: WHERE Id>@LastId
  19:  
  20: WHILE (@currentId IS NOT NULL)
  21: BEGIN
  22:  
  23:     SELECT @statName=StatName, @tableName=TableName FROM @dtaStats WHERE Id=@currentId
  24:     
  25:     PRINT 'DROP STATISTICS ' + @tableName + '.' + @statName
  26:     
  27:     EXEC('DROP STATISTICS ' + @tableName + '.' + @statName)
  28:     
  29:     SET @lastId = @currentId 
  30:     SET @currentId=null
  31:  
  32:     SELECT @currentId=MIN(Id)
  33:     FROM @dtaStats 
  34:     WHERE Id>@LastId
  35:  
  36: END
  37:  
  38:  

Then drop the generated indexes:

   1:  
   2: DECLARE @dtaIndex AS TABLE(Id INT IDENTITY(1,1), IndexName VARCHAR(255), TableName VARCHAR(255))
   3: DECLARE @currentId AS INT
   4: DECLARE @lastId AS INT
   5:  
   6: DECLARE @IndexName VARCHAR(255)
   7: DECLARE @tableName VARCHAR(255)
   8:  
   9: INSERT INTO @dtaIndex(IndexName,TableName)
  10:     SELECT name, OBJECT_NAME(OBJECT_ID) AS TABLENAME
  11:     FROM sys.indexes
  12:     WHERE name like '_dta_index_%'
  13:     
  14: SET @lastId=0
  15: SET @currentId=NULL
  16:  
  17: SELECT @currentId=MIN(Id)
  18: FROM @dtaIndex 
  19: WHERE Id>@LastId
  20:  
  21: PRINT @currentId
  22:  
  23: WHILE (@currentId IS NOT NULL)
  24: BEGIN
  25:  
  26:     SELECT @IndexName=IndexName, @tableName=TableName FROM @dtaIndex WHERE Id=@currentId
  27:     
  28:     PRINT 'DROP INDEX ' + @IndexName + ' ON ' + @tableName
  29:     
  30:     EXEC('DROP INDEX ' + @IndexName + ' ON ' + @tableName)
  31:     
  32:     SET @lastId = @currentId 
  33:     SET @currentId=NULL
  34:  
  35:     SELECT @currentId=MIN(Id)
  36:     FROM @dtaIndex 
  37:     WHERE Id>@LastId
  38:  
  39: END
  40:  
  41:  

The database is now back to a pre-DTA’ed state!

Saturday, 30 October 2010

Infrastructure: Replacing a Dead RAID 5 Drive in QNAP NAS

It’s been a while since I last posted – it’s been crazy busy at work – and this is a completely non-.Net related post (sorry).  However, like many of you (I’m  sure) I’ve got a personal SVN repository (where I keep my .Net code, so there is a tenuous link!) which is hosted on my QNAP 419P NAS Drive on a RAID5 volume across 4 Hot Swappable Disks. 

For the non-technical, this is a little whizzy box with tonnes of space and can withstand a single complete hard disk failure without loosing data. 

Recently one of the disk drives failed in the volume (completely dead) and whilst my data’s intact and everything carried on as best as it could, I needed to replace the drive.  I was completely under the impression that this operation went along the lines of:

  1. Remove Dead Drive
  2. Insert New Disk
  3. Click ‘Repair’ on RAID volume in config
  4. Wait for repair to complete
  5. PROFIT!!

And with most corporate RAID solutions that’s exactly what happens.  However, after completing Step 2 of my plan I discovered that the QNAP didn’t recognise the new disk at all and the Repair button was greyed out.

WTF?!?

Well, it turns out the QNAP’s software raid manager can’t automatically recover from actual dead disks, only disks incorrectly flagged as faulty.  Genius.

After a bit of Googling, I found that ‘all’ I needed to do was duplicate the partition table from one of the remaining live disks to the new disk and hopefully the low-level raid manager will them automatically rebuild the array.

So the actual steps are:

  1. Replace disk with new one
  2. SSH into QNAP box with Administrator priveleges
  3. Get the partition settings from fdisk (type: fdisk –l)
  4. Identify a live disk in the output, in my case these were /dev/sda, /dev/sdb, /dev/sdc  (there should also have been /dev/sdd – but that was the dead one):

       Device Boot      Start         End      Blocks   Id  System
    /dev/sdc1   *           1          66      530113+  83  Linux
    /dev/sdc2              67         132      530145   82  Linux swap / Solaris
    /dev/sdc3             133      243138  1951945695   83  Linux
    /dev/sdc4          243139      243200      498015   83  Linux

  5. Now to replicate this config to the new drive (/dev/sdd).  My commands are in bold italics:


    [~] # fdisk /dev/sdd
    Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
    Building a new DOS disklabel. Changes will remain in memory only,
    until you decide to write them. After that, of course, the previous
    content won't be recoverable.


    The number of cylinders for this disk is set to 243201.
    There is nothing wrong with that, but this is larger than 1024,
    and could in certain setups cause problems with:
    1) software that runs at boot time (e.g., old versions of LILO)
    2) booting and partitioning software from other OSs
       (e.g., DOS FDISK, OS/2 FDISK)
    Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

    Command (m for help): n
    Command action
       e   extended
       p   primary partition (1-4)
    p
    Partition number (1-4): 1
    First cylinder (1-243201, default 1):1
    Using default value 1
    Last cylinder or +size or +sizeM or +sizeK (1-243201, default 243201): 66

    Command (m for help): n
    Command action
       e   extended
       p   primary partition (1-4)
    p
    Partition number (1-4): 2
    First cylinder (67-243201, default 67):67
    Using default value 67
    Last cylinder or +size or +sizeM or +sizeK (67-243201, default 243201): 132

    Command (m for help): n
    Command action
       e   extended
       p   primary partition (1-4)
    p
    Partition number (1-4): 3
    First cylinder (133-243201, default 133):133
    Using default value 133
    Last cylinder or +size or +sizeM or +sizeK (133-243201, default 243201): 243138

    Command (m for help): n
    Command action
       e   extended
       p   primary partition (1-4)
    p
    Selected partition 4
    First cylinder (243139-243201, default 243139):243139
    Using default value 243139
    Last cylinder or +size or +sizeM or +sizeK (243139-243201, default 243201): 243200

  6. Next to mark /dev/sdd1 as bootable Command (m for help): a
    Partition number (1-4): 1

  7. Then change partition 2 to ‘Linux Swap / Solaris’ format Command (m for help): t
    Partition number (1-4): 2
    Hex code (type L to list codes): 82
    Changed system type of partition 2 to 82 (Linux swap / Solaris)
  8. Finally, save the new partition table Command (m for help): w
    The partition table has been altered!
  9. Ctrl-C to exit fdisk
  10. Eject the new disk and reinsert
  11. The new disk is recognised and rebuilding begins
  12. Wait for rebuild to complete
  13. PROFIT!!

I’m aware that on a full linux distribution there are better approaches than this, but on the QNAP with it’s subset of commands this gets the job done reasonably well.

Monday, 20 September 2010

Security: Protect against POET Attacks with Custom Errors!

There’s been a big deal made of a serious security flaw in ASP.Net which potentially affects a lot of .Net sites, that allows a 3rd Party to trick ASP.Net into serving sensitive files within a web application folder.  Microsoft have released official advise on how to temporarily patch the problem which revolves around forcing Error  and Page Not found pages to return the same status page.  This would need to stay in place until a permanent fix is released.

This workaround clearly introduces an usability issue, which client may not accept.

Fortunately a quick amend to my Custom Error Module can secure your site against the attack with minimal impact to usability. 

   1: using System;
   2: using System.Web;
   3: using System.Net;
   4: using System.Collections.Generic;
   5: using System.Configuration;
   6: using System.Web.Configuration;
   7:  
   8:  
   9: namespace MartinOnDotNet.Website.Support
  10: {
  11:     /// <summary>
  12:     /// Handles errors in an SEO friendly manner
  13:     /// </summary>
  14:     public class SeoErrorLoggingModule : IHttpModule
  15:     {
  16:  
  17:         private static System.Random random = new Random((int)DateTime.Now.Ticks);
  18:  
  19:         private const int MaxDelay = 500;
  20:  
  21:         private static CustomErrorsSection customErrors = WebConfigurationManager.GetSection("system.web/customErrors") as CustomErrorsSection;
  22:  
  23:         /// <summary>
  24:         /// Called when [error].
  25:         /// </summary>
  26:         /// <param name="sender">The sender.</param>
  27:         /// <param name="e">The <see cref="System.EventArgs"/> instance containing the event data.</param>
  28:         protected virtual void OnError(object sender, EventArgs e)
  29:         {
  30:             HttpApplication application = (HttpApplication)sender;
  31:             HttpContext context = application.Context;
  32:             if (context != null && context.AllErrors != null)
  33:             {
  34:                 foreach (Exception ex in context.AllErrors)
  35:                 {
  36:                     ex.Data["RawUrl"] = context.Request.RawUrl;
  37:                     HttpException hex = ex as HttpException;
  38:                     if (hex != null && hex.GetHttpCode() == (int)HttpStatusCode.NotFound)
  39:                     {
  40:                         Logging.Logger.LogWarning(string.Format(System.Globalization.CultureInfo.InvariantCulture, "Requested File Not Found {0} ({1})", context.Request.RawUrl, context.Request.Url));
  41:                     }
  42:                     else
  43:                     {
  44:                         Logging.Logger.Log(ex);
  45:                     }
  46:                    
  47:                 }
  48:             }
  49:             HttpException httpException = context.Error as HttpException;
  50:             context.Response.Clear();
  51:             if (httpException != null && !IsResourceRequest(context.CurrentHandler))
  52:                 context.Response.StatusCode = httpException.GetHttpCode();
  53:             else
  54:                 context.Response.StatusCode = (int)HttpStatusCode.InternalServerError;
  55:             if (((context.IsCustomErrorEnabled && !context.Request.Browser.Crawler) || IsResourceRequest(context.CurrentHandler) )
  56:                 && !IsAnErrorPage(context.Request.RawUrl))
  57:             {
  58:                 System.Threading.Thread.Sleep(TimeSpan.FromMilliseconds(_random.Next(MaxDelay)));
  59:                 context.ClearError();
  60:                 string path = GetPathForError(context, (HttpStatusCode)context.Response.StatusCode);
  61:                 if (!string.IsNullOrEmpty(path))
  62:                 {
  63:                    if (CustomErrors.RedirectMode == CustomErrorsRedirectMode.ResponseRedirect && !IsResourceRequest(context.CurrentHandler) )
  64:                    {
  65:                        context.Response.Redirect(path, true);
  66:                    }
  67:                    else
  68:                    {
  69:                        context.RewritePath(path);
  70:                    }
  71:                 }
  72:             }
  73:         }
  74:  
  75:         /// <summary>
  76:         /// Determines whether current request is to a resource handler
  77:         /// </summary>
  78:         /// <param name="handler">The handler.</param>
  79:         /// <returns>
  80:         ///     <c>true</c> if [is resource request] [the specified handler]; otherwise, <c>false</c>.
  81:         /// </returns>
  82:         protected virtual bool IsResourceRequest(IHttpHandler handler)
  83:         {
  84:             return handler != null
  85:                 &&
  86:                 (typeof(System.Web.Handlers.AssemblyResourceLoader).IsInstanceOfType(handler)
  87:                 || typeof(System.Web.Handlers.ScriptResourceHandler).IsInstanceOfType(handler));
  88:  
  89:         }
  90:  
  91:         /// <summary>
  92:         /// Gets the path for error.
  93:         /// </summary>
  94:         /// <param name="current">The current.</param>
  95:         /// <param name="status">The status.</param>
  96:         /// <returns></returns>
  97:         protected virtual string GetPathForError(HttpContext current, HttpStatusCode status)
  98:         {
  99:             foreach (CustomError ce in customErrors.Errors)
 100:             {
 101:                 if (ce.StatusCode == (int)status) return ce.Redirect;
 102:             }
 103:             return customErrors.DefaultRedirect;
 104:         }
 105:  
 106:         /// <summary>
 107:         /// Determines whether the given path (RawUrl) is an error page itself
 108:         /// </summary>
 109:         /// <param name="path">The path.</param>
 110:         /// <returns>
 111:         ///     <c>true</c> if [is an error page] [the specified path]; otherwise, <c>false</c>.
 112:         /// </returns>
 113:         protected virtual bool IsAnErrorPage(string path)
 114:         {
 115:             if (ErrorPages != null)
 116:             {
 117:                 foreach (string s in ErrorPages)
 118:                 {
 119:                     if (path.IndexOf(s, StringComparison.OrdinalIgnoreCase) > -1) return true;
 120:                 }
 121:             }
 122:             return false;
 123:         }
 124:  
 125:         /// <summary>
 126:         /// Gets the error pages.
 127:         /// </summary>
 128:         /// <value>The error pages.</value>
 129:         protected virtual IEnumerable<string> ErrorPages
 130:         {
 131:             get
 132:             {
 133:                 
 134:                 foreach (CustomError ce in customErrors.Errors)
 135:                 {
 136:                     yield return ce.Redirect;
 137:                 }
 138:                 yield return customErrors.DefaultRedirect;
 139:             }
 140:         }
 141:  
 142:        /// <summary>
 143:        /// Disposes of the resources (other than memory) used by the module that implements <see cref="T:System.Web.IHttpModule"/>.
 144:        /// </summary>
 145:        public void Dispose()
 146:        {
 147:            //clean-up code here.
 148:        }
 149:  
 150:        /// <summary>
 151:        /// Initializes a module and prepares it to handle requests.
 152:        /// </summary>
 153:        /// <param name="context">An <see cref="T:System.Web.HttpApplication"/> that provides access to the methods, properties, and events common to all application objects within an ASP.NET application</param>
 154:        public void Init(HttpApplication context)
 155:        {
 156:            // Below is an example of how you can handle LogRequest event and provide 
 157:            // custom logging implementation for it
 158:            context.Error += new EventHandler(OnError);
 159:        }
 160:  
 161:  
 162:     }
 163: }

The amendments hinge on the fact that the exploit only really affects the WebResource.axd and ScriptResource.axd so any error relating from these handlers is automatically given an 500 status (Internal Server Error) and treated as a normal error.  This is an acceptable compromise for me as all references to these handlers should be programmatically generated and by your site and therefore ‘correct’.

As per, Scott Gu’s recommendation I’ve added a random <500ms delay to the processing of all errors to help muddy the waters and added support for the ResponseRewrite property on the CustomErrors element.