shutupandcode.net

I’ve seen some dramatic failures because people have waited till the end. Only until after they were ‘done coding’ did they get the shock that it takes 30 seconds for their application to return “Hello World” and that Hello World call took 99% of cpu utilization and 600MB of ram.

If there was any tip/trick to writing high performance applications it would be that performance goals should constantly be mentioned, examined, and logged from day one. I don’t know how many times I have seen teams crumble as deadlines approached to see their applications fail any reasonable performance metric.

The problem for them? THEY NEVER MEASURED IT IN THE FIRST PLACE.

If you reach the end, and you find yourself in this situation. Just stop what you are doing and examine your choices:

A.) Start throwing spaghetti at a wall and see what sticks. Hope above all else you’ll find some glaringly obvious design flaw that caused your performance problems, because there isn’t a chance in hell you’re going to be able to correct a performance problem because of architecture or worse, small little choices sprinkled in a huge code-base which may or may not have inverse relationships to each other.

B.) Re-write your application. Start with the smallest unit of code and start porting over the “bad” code as you go along. Continuously Integrating and evaluating your performance along the way keep a watchful eye on that performance #.

It’s sad really…. No I really mean it… It’s very sad…..

I’ve seen so many projects also go with choice A. Thinking (read: more like hoping) that somehow, someway, you could make a code base which was never monitored for performance, a codebase of say 30,000+ lines of code, meet performance expectations.

Out of those projects I’ve seen go with choice A, I’ve never seen one come up with a good solution. I’ve seen workarounds (see IT dictionary), and I’ve seen things crash and burn, but I’ve never seen a project that went with choice A succeed. It just will never happen… there is just too many variables unaccounted for.

So if your on a team with a poorly performing application scheduled to go into production….

Realize you have already missed the boat, accept it, and move on. Just don’t make your customers pay the price.

Consider this, you are a hosting company hosting many different customer websites. Your customers pay anywhere between $50.00 to $100.00 a month to have a shared server host there asp.net applications. You business model depends on easily providing the exact same level of support and features to your clients that they could otherwise get if they did it themselves, you can do so at a reduced rate because you cookie-cut their commonly requested features. The key is that you provide “the exact same” level of support and features.

The inability for you to provide something to them means you lose a piece of your market share and credibility. With hosting companies on razor thin profit margins, you rely on lots of low to mid-range consumers, and I mean lots of them. You’re effectively a wal-mart relying on the poor masses to accumulate large revenue through low over-head.

Now throw in a rather large WCF limitation…..


This collection already contains an address with scheme http. There can be at most one address per scheme in this collection.
Parameter name: item


Exception Details:

System.ArgumentException: This collection already contains an address with scheme http. There can be at most one address per scheme in this collection.
Parameter name: item


Source Error:
An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

Stack Trace:

[ArgumentException: This collection already contains an address with scheme http. There can be at most one address per scheme in this collection.
Parameter name: item]
System.ServiceModel.UriSchemeKeyedCollection.InsertItem(Int32 index, Uri item) +4564833
System.Collections.Generic.SynchronizedCollection`1.Add(T item) +56
System.ServiceModel.UriSchemeKeyedCollection..ctor(Uri[] addresses) +120
System.ServiceModel.ServiceHost..ctor(Type serviceType, Uri[] baseAddresses) +145
System.ServiceModel.Activation.ServiceHostFactory.CreateServiceHost(Type serviceType, Uri[] baseAddresses) +28
System.ServiceModel.Activation.ServiceHostFactory.CreateServiceHost(String constructorString, Uri[] baseAddresses) +323
System.ServiceModel.HostingManager.CreateService(String normalizedVirtualPath) +516
System.ServiceModel.HostingManager.ActivateService(String normalizedVirtualPath) +31
System.ServiceModel.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath) +498


Hosting multiple sites in Windows 2000-2008 Server typically means that your using IIS to host multiple sites on the same IP Address using host header filters or at worst port level filters.Now enter the above WCF “feature” (aka bug). The simple ability for you to host multiple WCF services for different hosted addresses in IIS is defunct. What does this mean to you? Well if your on a hosted environment (either internal cooperate network, or a classic joe’s hosting company) and you use IIS you are going to have to follow Rob Zelt instructions and it requires a code change!

Workaround #1: Customize your .svc file and use a custom WCF factory.

< %@ServiceHost language=c# Debug="true" Service="Microsoft.ServiceModel.Samples.CalculatorService"
Factory="Microsoft.ServiceModel.Samples.CustomHostFactory" %>

class CustomHostFactory : ServiceHostFactory
{
protected override ServiceHost CreateServiceHost(Type serviceType, Uri[] baseAddresses)
{
CustomHost customServiceHost =
new CustomHost(serviceType, baseAddresses[1]);
return customServiceHost;
}
}

class CustomHost : ServiceHost
{
public CustomHost(Type serviceType, params Uri[] baseAddresses)
: base(serviceType, baseAddresses)
{ }
protected override void ApplyConfiguration()
{
base.ApplyConfiguration();
}
}

The reason for this mess is that ServiceHost uses the protocol (string value “http”, “https”) as the key for the BaseAddresses collection. This likely looks something like this inside the core .NET library.

IDictionary<string , string[]> _protocols = new Dictionary</string><string , string[]>();

private void RegisterProtocol(string protocol, string[] addresses) {

// whoops, we forgot to check if _protocols already had the key "protocol".... shucks...

_protocols.Add(protocol, addresses);
}

Somebody had reported that this has been fixed in what was “Orcas” but last I checked it hasn’t.

There is one other workaround (care-of: Jon Flander’s Blog) which involves changing the way you actually host your sites. Albeir this is not something feasible for a hosting provider.

Workaround #2: A way to solve this problem is to setup multiple web sites in IIS (rather than having one site that has multiple Host entries) - and point all the websites to the same directory.

This is, all in all, a great annoyance which will hopefully be fixed in later releases of .NET.

I really hate it. No, you have no idea. I really despise it….

It’s that crusty, old, smelly, and repugnant idea that everything… and I mean everything…. must be in a database. It’s a profuse nauseating waste of effort and time that just wont die. Don’t get me wrong, databases were great, they had a purpose, filling a gap between COBOL programs (that makes schema changes as easy as mixing oil and water) and well… anything else at the time.

I know, I know, all the other alternatives to-date are either not ‘enterprisey’ enough, or have as much performance association that electric hybrids have to every self-indulgent quadruple billionaire 42 year old looking for their next ultra-performance car. It just doesn’t register, no matter how much faster the electric is compared to a Lamborghini Guirado. It’s that perception that bothers me the most; and it is that perception that causes you to receive a slightly grotesque distortion of your co-workers face when you mention not using SQL and a database to store your data.

All that being said, databases are in your server room. They sit there, stunted, childish, semi-retarded things. But hey, lets admit it, we’re all probably working on a project right now using SQL Server 2005. And given that we were are forced to play with a semi-retarded child that eats sand, maybe we can put a bag over his/her head so we don’t have to pay attention to it.

Microsoft makes a-lot of money selling SQL Server, and they realize we (coders) don’t like it. They have at least made the effort in 2005 to add a new data-type with native XPATH, and with this little bit we can at least tolerate it, or at least not look at it so often. How might you ask?

Well consider you are creating a simple authentication process with your website. You want users to have multi-factored authentication. You know, where you get an ugly, plain, and generic picture that you are supposed to memorize to magically prevent you from being the victim of phishing schemes as well as some random security questions.

If your me, you start with your domain objects (contracts). And you come up with some .NET classes that serialize to some xml.
So now, you’ve got some options:

1. Store your data in an in memory object and be done with it.
2. Spend the next 2 weeks arguing with a pretentious DBA about a database schema that doesn’t comply with 5th normal form.

So given you are likely here because you can’t choose #1, how do we minimize the impedance-mismatch between our code and the relational database? Answer? Serialize to XML and ‘talk’ to the database using XML.

Going with our XML example (remember generated from XML Serialized C# class). We can use the serialized class and create a stored proc that takes in the serialized XML and maps it to the corresponding relational database. Forget updates, just go with full delete / then update. Example SQL 2005 parsing is shown below.

declare @xml as xml
set @xml = '... string contents of xml file ...'

-- user information
select
doc.value('@Attribute', 'varchar(max)') 'attribute'
, doc.query('Element').value('.', 'varchar(32)') 'Element'
, doc.query('Element/Nested').value('.', 'varchar(32)') 'ElementNested'
, doc.query('data(Element/@Id)').value('.', 'varchar(64)') 'NestedAttributeValue'
from
@xml.nodes('//Root/Items') tbl (doc)

What we are effectively doing is limiting the touch-points between the code and the database. Seperations-of-Concerns says that my objects shouldn’t care about their storage, and that my storage shouldn’t care about my objects. With the above example we would basically have two methods for our data access. Create, and Delete, with Read (Query) being the second part of this exploration and update being a full delete, with a full create.

Next time we can explore a database design, that is both normalized and allows for quick and efficient lookups of information while still allowing for a rich object model that can use the criteria (specification) pattern to deliver rich searching to your application without spreading database concerns throughout your application.

Just a quick little post today regarding how you can do some fairly simple color transformations to produce some visual ‘charting’ of data using html. This example stems from an experiment to simulate the StatsPress module that can be included in WordPress. The outcome of this is a color transformation class that can be used to perform some nice color ranges in both HEX (HTML) and System.Drawing.Color. Below is a screenshot of a produced graph. Attached is FxColor Source Code and below is the core class for your review.

using System;
using System.Drawing;
using System.Collections.Generic;

/// <summary>
/// Provides a means to create a color mixer that shifts colors
/// from one color matrix to another.
/// </summary>
public class FxColorMixer
{
int red, green, blue;
int delta;
int iterations;

/// <summary>
/// Initializes a new instance of the <see cref="FxColorMixer"/> class.
/// </summary>
/// <param name="red">The starting red color.</param>
/// <param name="green">The starting green.</param>
/// <param name="blue">The starting blue.</param>
/// <param name="iterations">The number of iterations to produce.</param>
public FxColorMixer(int red, int green, int blue, int iterations)
{
if (iterations < = 0) throw new ArgumentOutOfRangeException("iterations must be greater than 0");

this.red = red;
this.green = green;
this.blue = blue;
this.delta = (int)Math.Round((double)250 / (double)iterations, 0);
this.iterations = iterations;
}

/// <summary>
/// Gets the color at the specified index from the number of iterations specified.
///
/// <param name="index">The index.</param>
/// <param name="sRed">The s red.</param>
/// <param name="sGreen">The s green.</param>
/// <param name="sBlue">The s blue.</param>
/// <returns></returns>
public FxColor GetColor(int index, double sRed, double sGreen, double sBlue)
{
if (index > iterations) throw new ArgumentOutOfRangeException("index is past number of iterations expected.");

double vBlue = (delta * index) * sBlue;
double cBlue = blue + vBlue;

double vGreen = (delta * index) * sGreen;
double cGreen = green + vGreen;

double vRed = (delta * index) * sRed;
double cRed = red + vRed;

cRed = Math.Max(0, Math.Min(255, cRed));
cGreen = Math.Max(0, Math.Min(255, cGreen));
cBlue = Math.Max(0, Math.Min(255, cBlue));

return new FxColor(Color.FromArgb((int)cRed, (int)cGreen, (int)cBlue));
}
}

/// <summary>
/// Represents a adapter over the <see cref="Color"/> class.
/// </summary>
public class FxColor
{
Color c;

/// <summary>
/// Initializes a new instance of the <see cref="FxColor"/> class.
/// </summary>
/// <param name="c">The c.</param>
internal FxColor(Color c)
{
this.c = c;
}

/// <summary>
/// Offsets the current color by the vectors specified.
/// </summary>
/// <param name="vRed">The displacement of red.</param>
/// <param name="vGreen">The displacement of green.</param>
/// <param name="vBlue">The displacement of blue.</param>
/// <returns>Returns a new color from the displacement.</returns>
public FxColor Offset(int vRed, int vGreen, int vBlue)
{
int r = Math.Max(0, Math.Min(255, c.R + vRed));
int g = Math.Max(0, Math.Min(255, c.G + vGreen));
int b = Math.Max(0, Math.Min(255, c.B + vBlue));
return new FxColor(Color.FromArgb(r, g, b));
}

/// <summary>
/// Gets the value.
/// </summary>
/// <value>The value.</value>
public Color Value
{
get
{
return c;
}
}

/// <summary>
/// Gets the HEX value of this color.
/// </summary>
/// <value>The HEX.</value>
public string HEX
{
get
{
return ColorTranslator.ToHtml(c);
}
}
}

You’ve been there, working on a large or small project. Discovering things, learning new things, applying your new knowledge to get further and surpass your previous achievements. And it’s that brilliant, thunderous, and blinding bolt of inspiration that gives way to that beautiful and elegant piece of knowledge, simplistic yet profound, that keeps you moving.

How does this tie into working on a large ‘SOA’ based project at a major financial institution using .NET technologies like WCF (Microsoft Windows Communication Foundation) and ASP.NET? Most of these services just provide lookup information based on a set of criteria supplied to them, so they don’t contain anything more than simplified domain objects, no frills, no rules, no update functionality. Hardly anything interesting…

But yet these services being created are responsible for wading through distributed systems from Unix, to dozens of Windows SQL Server Databases, to CGI calls to mainframe pathways. Their goal, at the end of the day, is to provide an ‘enterprise’ domain model of the organization’s business, and also provide a new web application to be created to provide clients some information about their accounts. (Think Bank Of America, Charles Swabb).

On the surface hardly anything of the above seems interesting, or fun to code, but then, diving deeper we find the patterns, the ability to expand our technical skill set while improving the quality of our delivered code.

With that, this post starts long running series on ‘Explorations in Code’, and ‘Explorations in the Software Business’. Two parallel long running commentaries about my work and the insights I get through the project.

To wet the appetite the following posts will be made in due time:

• Parallel and Distributed Processing using commodity PC’s in .NET (MapReduce and F(x) algorithms)
• WCF Tips and Tricks
• .NET Reflection, Deep Dive
• Add-In / Plug-In Framework
• AVP Design Pattern (Assembled-View-Pattern) - Eat your heart out MVP
• JavaScript, How I Heart Thee
• Practical REST
• CSS, XHTML, JavaScript, and the new web.