Smelser.NET

In some high-performance scenario’s you may need to create objects at a higher rate than previous methods can support. This could also be true for scenario’s where the object creation processes involve a large number of steps before the object is considered fully constructed and ready for use. This can often be the case where surges of requests cause applications to breach Quality of Service (QoS) thresholds.

In cases such as this, a background object builder can help offset the expense of construction and provide the host some additional mitigation of its overall capacity during surges. Here is a sample implementation which will pre-build a queue of instances and will refresh its internal queue when the size falls below a configurable threshold.

Lets take a look at how we might use this. There are two basic usage scenarios that I can think of...

Activator Object Construction

First define an instance of the BackgroundActivator. In doing so you can specify the size of queued instances, and the size of the queue at which the it should be refilled.

Once we have an instance of the activator we can just call the CreateInstance method any time we need an instance of the object

Custom Object Construction

This scenario is ideal if your construction requirements are a bit more involved. In this case, a function is supplied which is called to perform the underlying construction.

This is a pattern I use within my IoC Container and can really provide that little extra boost when you need it. Use in good health. :-)

Tags: performance, design patterns

Despite the title, I'm actually a big fan of Log4Net. While powerful, Log4Net can become the bottleneck for some applications so we need to explore these issues a little more closely so you can configure Log4Net in a way that’s optimal for your system. Identifying this as a potential performance bottleneck is relatively straightforward. If you have a repeatable performance test, profile the application with your normal production log configurations, and then again with all your logging disabled. If the change in performance is dramatic, then you may want to consider a new strategy.

Check logging levels before logging

To avoid repeating information from the authors of this component… as a general rule you should always test if the target log level is enabled before you actually log anything. This will offset some hidden costs which the vendor’s article details.
 
More information can be found on the vendor’s site (http://logging.apache.org/log4net/release/manual/internals.html).

Using the RollingFileAppender

As a file I/O-bound resource, this Appender can block your application from doing real work while the Appender flushes its buffer to disk. Profiling your application will uncover this and there are a couple options to mitigate this expense.

Mitigation strategies

• Disable the immediate flush option. According to the Log4Net documentation, this can result in performance increases of 15-20% but has a risk that the final buffer may not be flushed to disk if the host process crashes

Using the AdoNetAppender

As a database I/O-bound resource, this Appender can block your application from doing real work while the Appender flushes its buffer to the database. If you analyze this closely, you’ll see that the native Appender will execute a Stored Procedure or SQL command for each and every log entry to be sent to the database. Unlike the RollingFileAppender, buffering will never offset the cost of these database interactions. As a result, more invasive strategies will be required.

Mitigation strategies

• Implement a custom Appender that inherits from AdoNetAppender and override the SendBuffer(LoggingEvent[]) method. At this point you can then change the implementation to suite your performance demands while preserving all the base class functionality such as managing connections, etc… Here are a few ideas on what you could implement:
  • Write to shared memory and offload to an external process to perform write-behind
  • Perform bulk inserts into the database
  • Batch multiple log entries into a single Stored Procedure call
  • Queue to an MSMQ and use an external process to perform write-behind
  • Etc…
• Avoid using custom parameter formatters. Under the covers many of the underlying PatternConverters result in the creation of StringBuilders which can be quite expensive. If you’re building a custom implementation of this Appender, it may be more performant to handle this using different patterns

Hope this helps you find that little something extra and find performance that you need. :-)

Tags: performance, design patterns

When authoring recursive routines there are special memory and performance considerations to be made. Firstly, authors must consider the overall size the call Stack could reach given the point in the Stack where the call starts and the number of Stack Frames that could be added as a result of the recursion. In doing so, you can estimate some key factors.

1. What is the likelihood of a StackOverflowException?
2. How much memory will be required to support all the Stack Frames added by the recursive call?

With just these two elements, we can assert a couple performance implications:

• If your recursion is expected to create a large number of Stack Frames, then your product may suffer from the direct impact of managing a large chunk of memory for the Stack and the indirect expense of collapsing the recursion results when the recursion is completed. In this case, optimizations may be valuable and will very likely result in an easily measureable performance advantage
• In cases, where you’re creating very shallow recursions (e.g. few Stack Frames) this is rarely a concern unless the state on the stack is very large

Assuming you need to optimize, you can try and achieve a Tail call (a.k.a. tail-recursive call). Before we go on, let’s define more precisely what a Tail Call is.

In computer science, a tail call is a subroutine call that happens inside another procedure and that produces a return value, which is then immediately returned by the calling procedure. The call site is then said to be in tail position, i.e. at the end of the calling procedure. If a subroutine performs a tail call to itself, it is called tail-recursive. This is a special case of recursion.

Tail calls are significant because they can be implemented without adding a new stack frame to the call stack. Most of the frame of the current procedure is not needed any more, and it can be replaced by the frame of the tail call, modified as appropriate (similar to overlay for processes, but for function calls). The program can then jump to the called subroutine. Producing such code instead of a standard call sequence is called tail call elimination, or tail call optimization.” --- Wikipedia

As explained in the definition, tail calls will prevent your recursion from producing any more than two Stack Frames. This is because the second Frame will continually be reused for each point of recursion.

Unfortunately, achieving tail calls in .NET is somewhat difficult. There are two commonly used patterns for trying to achieve this:

1. Modify your method structure to make it eligible for tail-elimination optimizations by the C# compiler. To determine if this has already happened for your method you simply need to review the IL and see if the following IL is present near the end of your method
      
   IL_001a: tail
   
   If the tail IL is already present, then you’re all done… well done! If not, then making your method eligible simply involves modifying the method so that the recursive call is the last statement in the method
     
   private int ARecursiveMethod(int arg)
   {
       //TODO: Do work
       return ARecursiveMethod(arg);
   }
2. Force a tail call. Unfortunately, the compiler’s exception list for automatically applying this optimization is very long. Consequently there are a number of reason’s this will not occur automatically. As it turns out, the list of reasons is largely a result of just-in-case scenarios where they want to avoid breaking your method. Although a more advanced set of steps is required, you can force tail calls to occur if your process requires it. Here is what you need to do:

• Compile the parent assembly as normal
• Extract the IL with ILDASM
• Modify the IL to insert the tail instruction
• Reassemble the assembly with ILASM
• Test, test, test…

Note: This can all be automated but obviously carries some risk. In my experience, this should occur in CI between build time and Unit Test runs

Have Fun!

Tags: performance, design patterns

The JIT compiler will; unless told otherwise; apply a number of optimizations at runtime to address both minor and major inefficiencies in the code we write. Examples include the following:

• Constant Folding
• Constant and Copy Propagation
• Method Inlining
• Code Hoisting and Dominators
• Loop Unrolling
• Common SubExpression Elimination
• Enregistration
• And others…

Generally we don’t need to think about these details too much, although where hyper-performance concerns exist we should. The reason is because our coding patterns can lead to situations where the JIT decides to completely skip optimizations to ensure it doesn’t break the code we’ve written. Let’s take Method Inlining for example. Here is a list of JIT rules that guide its decision to optimize:

… Method Inlining
There is a cost associated with method calls; arguments need to be pushed on the stack or stored in registers, the method prolog and epilog need to be executed and so on. The cost of these calls can be avoided for certain methods by simply moving the method body of the method being called into the body of the caller. This is called Method In-lining. The JIT uses a number of heuristics to decide whether a method should be in-lined. The following is a list of the more significant of those (note that this is not exhaustive):

• Methods that are greater than 32 bytes of IL will not be inlined
• Virtual functions are not inlined
• Methods that have complex flow control will not be in-lined. Complex flow control is any flow control other than if/then/else; in this case, switch or while
• Methods that contain exception-handling blocks are not inlined, though methods that throw exceptions are still candidates for inlining
• If any of the method's formal arguments are structs, the method will not be inlined. …

As developers, there are a few basic behaviors we should follow to ensure we can take advantage of these optimizations:

• Keep method size small (e.g. < 32 bytes of IL)
• Keep the contents of a try-catch-finally block small by moving the code to be executed in the try and catch blocks to another method
• Don’t make members virtual unless you have a clear need to do so

Tags: design patterns, performance

When authoring high performance applications, the following generalized rules can be considered valuable for creating highly performant capabilities:

Share nothing across threads, even at the expense of memory

• Sharing across thread boundaries leads to increased preemptions, costly thread contention, and may introduce other less obvious expenses in L2 cache, and more
• When working with shared state that is seldom or never updated, give each thread its own copy even at the expense of memory
• Create thread affinity if the workload represents saga’s for object state, but keep in mind this may limit scalability within a single instance of a process

• Where possible, isolating threads is ideal

Embrace lock-free architectures

• The fewer locks the better, which is obvious to most people

• Understanding how to achieve thread-safety using lock-free patterns can be somewhat nuanced, so digging into the details of how the primitive/native locking semantics work and the concepts behind memory fencing can help ensure you have leaner execution paths

# dedicated long-running Threads == Number of processing cores

• It’s easy to just spin up another thread. Unfortunately, the more threads you create the more contention you are likely to create with them. Eventually, you may find your application is spending so much time jumping between threads; there is no time to do any real work. This is known as a ‘Live Lock’ scenario and is somewhat challenging to debug
• Test the performance of your application using different threading patterns on hardware that is representative of the production environment to ensure the number of threads you’ve chosen is actually optimal
• For background tasks that have more flexibility in how often they can be run, when, and how much work can be done at any given time, consider Continuation or Task Scheduler patterns and collapse them onto fewer (or a single) threads

• Consider using patterns that utilize the ThreadPool instead of using dedicated long-running threads

Stay in-memory and avoid or batch I/O where possible

• File, Database, and Network I/O can be costly
• Consider batching updates when I/O is required. This includes buffering file writes, batch message transmissions, etc…

• For database interactions, try using bulk inserts even if it’s only to temp tables. You can use Stored Procedures to signal submission of data, which can then perform ETL like functions in the database

Avoid the Heap

• Objects placed on the Heap carry with them the burden of being garbage collected. If your application produces a large number of objects with very short lives, the burden of collection can be expensive to the overall performance of your application
• Consider switching to Server GC (a.k.a multi-core GC)
• Consider switching to Value Types maintained on the call stack and don’t box them
• Consider reusing object instances. Samples of each of these can be found in the below Coding Guidelines

Use method-level variables during method execution and merge results with class-level variables after processing

• Using shared variables that are frequently updated can create inefficiencies in how the call stack is managed and how L2 Cache’s behave
• When working with relatively small variables, follow a pattern of copy-local, do work, merge changes
• For shared state that is updated frequently by multiple threads, be aware of ‘False Sharing’ concerns and code accordingly

Avoid polling patterns

• Blind polling can lead to inefficient use of resources and reduce a systems ability to scale or reduce overall performance. Where possible, apply publish-subscribe patterns

Know what things cost

• If you dig a little deeper into the Framework you may find some surprises with regard to what things cost. Take a look at Know What Things Cost.

Develop layers with at least two consumers in mind… Production & Tests

• Developing highly performant systems requires a fair amount of testing. As such, each new layer/component needs to be testable in isolation so that we can better isolate performance bottlenecks, measure thresholds and capacity, and model/validate behavior under various load scenarios
• Consider using Dependency Injection patterns to allow injection of alternate dependencies, mocks, etc…
• Consider using Provider patterns to make selection of separate implementations easier. It’s not uncommon for automated test systems to configure alternate implementations to help suite various test cases
  • Ex. Layers that replicate network instability, layers that accommodate Bot-users that drive change in the system, layers that replicate external resources with predictable behaviours, etc…

Tags: performance, design patterns, threading

When your working on high performance products you'll eventually find yourself creating some form of a Near-Cache. If things really need to go fast, you may start thinking about going lock-free. This may be perfectly reasonable, but lets explore this a little to see where it makes sense.

For the purposes of this article, a Near-Cache is defined as any collection of state from durably persistent mediums that is maintained in-process and services requests concurrently. This may be a direct projection of state from a single medium or a custom View of state from many different mediums.

Typically, the implementation of Near-Cache patterns fall into a couple broad categories

Write-Once/Read-Many

• As the name suggests this is a cache that is seldom written too, and spends most of its time servicing read/lookup requests. A common example would be maintaining an in-memory View of static (or seldom updated) state in a database to offset the expense of calling back to the database for each individual lookup

Volatile Read-Write

• This type of cache typically has data written-to or altered as frequently as its read. In many cases, caches of this type maintain state that is discovered through more dynamic processes, although it could also be backed against persistent or other semi-volatile mediums

The benefits and/or consequences of selecting either pattern varies depending on the needs of your application. Generally speaking, both approaches will save costs as compared to calling-out to more expensive external resources. In C# there are several patterns which could be used to create a Thread-Safe Near-Cache. Exploring two samples to demonstrate opposite ends of the spectrum will help reveal their relative merits. Along the way I'll identify some criteria that may be useful in determining which is the most appropriate for a given scenario.

The Full-lock Cache

This pattern is suitable for either Write-Once/Read-Many or Volatile Read-Write caches

Benefits

• The locking and Thread-Safety semantics are easily understood
• Unless you have high numbers of threads accessing the cache concurrently or have extremely high performance demands, the relative performance of this pattern can be quite reasonable
• This can be relatively easy to test and could be made easier with subtle extensions

Consequences

• The consequences of this cache pattern would be the full lock placed on each call. This can start to manifest as higher contention rates for applications that have large numbers of threads or fewer threads with very high performance demands

The Lock-Free Cache

This pattern is only suitable for a Write-Once/Read-Many cache

Benefits

• No locking is required
• Read performance of a cache is significantly improved
• This can be relatively easy to test and could be made easier with subtle extensions

Consequences

• Writes can be relatively slow when compared to other patterns. This is a result of reloading the entire cache, or cloning and then updating the existing cache

Variations

• The Hashtable and ConcurrentDictionary types work very well in place of the Dictionary. These perform almost the same as the Dictionary

Tags: design patterns, performance, threading

Many applications have a requirement to execute code on a recurring schedule. In cases such as this, there is a tendency to create dedicated background threads to perform this function. This pattern certainly works, but can carry some in-efficiencies when you have large numbers of threads that spend the vast majority of their life sleeping.
As an alternative, consider using an invocation callback pattern.

To implement this pattern we need to start with an interface to define the invoker.

Next we need to define the concrete implementation. Under the covers this uses the System.Timers.Timer which manages schedule-based invocations via the ThreadPool. This approach is lightweight, easy to manage, and testable in isolation.

Finally, lets take a look at how to use this.

Other Considerations / Selection Criteria

• The InvocationTimer implementation of this pattern runs out of the ThreadPool. Consequently, when the pool is under pressure, things may not fire exactly on schedule. If you have a strict schedule to keep, consider an alternate pattern
• The InvocationTimer is not blocked while waiting for Invoked members to complete an invocation. Consequently, the subscriber must manage concerns with regard to re-entrancy. The sample above shows one simple way to do this
• The InvocationTimer is backed by the IInvocationTimer interface to make it mockable. Doing so, allows this concern to be removed from consuming layers for more isolated testing

Nice!!!

Tags: performance, threading, design patterns

START to SaveSomething
    CALL Logger with 'Entering SaveSomething'

    BEGIN
        IF User Has Permission
            CALL PersistSomething
    EXCEPTION 
        CALL Logger with 'Procedure Failed'
    END

    CALL Logger with 'Exiting SaveSomething'
END SaveSomething

If you stand-back and squint your eyes, this will probably look a little familiar. This represents that all-to-familiar pattern where we blur the lines of responsibility because its the most convenient way to accomplish things like logging, permission checks, tracing, etc... Unfortunately, this can lead to things like the following making your code much harder to read, debug, etc...

#if (DEBUG)
  CALL Logger with 'Procedure Entered'
#endif

The slippery slope should be obvious, but the more important issue is that we've mixed completely separate areas of concern making our code more inflexible. In this case, authorization, logging, and entity specific domain logic are all jumbled together. Putting this specific; and possibly terrible; example aside... this may be reasonable in some cases, but there are patterns that can provide us more flexibility. Ideally we'd be able to aggregate concerns such as these without being so tightly coupled. In this case, we'll explore a combination of the Proxy Pattern and the Interceptor Pattern which are commonly found in Aspect-Oriented Programming (AOP). The combination of these patterns creates what is loosely defined as a Ghost Proxy Object, although some usages do not include interception. Lets start with a simple Domain Object with a simple operation.

public class DomainObject : IDomainObject
{
    private readonly IDal _dal;

    public DomainObject() : this(IoC.Resolve<IDal>())    { }

    internal DomainObject(IDal injectedDal)
    {
        _dal = injectedDal;
    }

    public virtual void SaveSomething(object thing)
    {
        _dal.SaveSomething(thing);
    }
}

The class is pretty straightforward... get an object, and then save it. From a Domain perspective, that is the only concern this component needs to have. Now lets wrap this in a proxy and add methods to wire-up externally supplied delegates. The actual code to wire-up the delegates is not really important and would be pretty trivial to implement. suffice it to say, registered delegates will be called before and after the proxied method is called. The goal is to have a class that represents the base DomainObject while providing us with opportunities to execute externally registered logic. Additionally, all this should be accomplished without any changes to; or accommodations by; the base type aside from it being inheritable.

public sealed class ProxyObject : DomainObject, IInterceptable
{
    /* The real domain object instance to be called */
    readonly IDomainObject _realInstance;

    static IDictionary<string, BeginAction[]> _beginActions = new Dictionary<string, BeginAction[]>();
    static IDictionary<string, EndAction[]> _endActions = new Dictionary<string, EndAction[]>();

    public ProxyObject() : this(IoC.Resolve<IDal>())    { }

    internal ProxyObject(IDal injectedDal)
    {
        _realInstance = new DomainObject(injectedDal);
    }

    public override void SaveSomething(object thing)
    {
        this.InvokeBeginMethod(ref _beginActions, "Void SaveSomething(System.Object)", new[] { thing });

        _realInstance.SaveSomething(thing);

        this.InvokeEndMethod(ref _endActions, "Void SaveSomething(System.Object)", new[] { thing }, null);
    }

    public void AddInterceptMethod(MemberInfo memberInfo, BeginAction beginAction, EndAction endAction)
    {   /* Extension Method for instances of IInterceptable */
        this.AddInterceptMethod(ref _beginActions, ref _endActions, memberInfo, beginAction, endAction);
    }

    public void RemoveInterceptMethod(MemberInfo memberInfo)
    {   /* Extension Method for instances of IInterceptable */
        this.RemoveInterceptMethod(ref _beginActions, ref _endActions, memberInfo);
    }
}

Lets start by calling the un-proxied DomainObject to see the basic behavior in action.

IDomainObject realObject = new DomainObject();
realObject.SaveSomething("Foo");

Calling an instance of the proxy would produce the same result, but we need to inject additional functionality such as logging, tracing, security, etc... To do this, we must first create an instance of the Proxy and register the Delegates which will be called before and after proxied methods are called.

MethodInfo methodToIntercept = typeof(IDomainObject).GetMethod("SaveSomething");
ProxyObject proxyObject = new ProxyObject();
proxyObject.AddInterceptMethod(methodToIntercept, LogBeginAction, LogEndAction);
proxyObject.AddInterceptMethod(methodToIntercept, PermissionCheckBeginAction, null);

Once registered, any calls to new instances of the proxy type will be wrapped with the intercepting delegate calls.

IDomainObject proxyDomainObject = new ProxyObject();
proxyDomainObject.SaveSomething("Foo");

As you can see this works fine, but this could be improved by using an IoC container. This gives us the flexibility to control this behavior externally, which provides a wide range of new options. In this case, my IoC Container returns an instance of ProxyObject which is mapped to the IDomainObject interface in an external configuration file.

IDomainObject proxyDomainObject2 = IoC.Resolve<IDomainObject>();
proxyDomainObject2.SaveSomething("Foo");

These patterns can provide some really interesting capabilities although; as you can see; they require some investment in additional code. These days, proxies such as this can be automatically created using Code Generation or Dynamic Proxy tools (here is a sample). Another important consideration is performance... Profile your application and you'll see more in memory which means more to be GC'd. In the end, this means things will be slower. Like anything else, this is all about options, so take a look and see where this pattern might fit into your tool bag. Enjoy...

Tags: side projects, design patterns, code generation

Recently I was working on a custom transport and at some point I realized my payload had grown to an unacceptable size. After some analysis I discovered that DateTime structures represented the bulk of the problem. On reflection this was completely obvious since the default binary representation of a DateTime is a long (e.g. 8 bytes). This allows us to store date representations across just over 10,000 years.

In my case; and probably many of yours; I don't need to work with such a wide range of dates. This means I can shrink my dates by half and still have a sufficient range of dates to meet the needs of most applications. 

To do this, we can take a lesson from the POSIX community which has been converting Unix Time to and from other formats for a long time. This is a simple pattern that allows us to reuse the built-in DateTime structures. What we need are methods to convert to and from our offset 4 byte representation.   

One of the keys to this is setting a common time offset. Above you will see that I set my offset to start at 01/01/2010. As long as this never changes, then you can adjust this as needed to meet your particular requirements. With these methods in place we can apply them in reading/writing to transports, stores, or whatever. Here is an example of writing a DateTime in its short form:

Here is an example of reading the offset bytes back into a normal DateTime object:

That's all there is to it... Apply this in good health. 

Tags: performance, design patterns

Some time ago I wrote about creating custom configuration sections as an alternative to using appSettings. At the time, the focus revolved largely around the need for hierarchal and strongly typed configuration points, neither of which are really a feature of sections like appSettings. Although I feel like I'm picking on the poor little appSettings a little bit, if they're used incorrectly they can really hurt products that demand really high performance. Lets look at a common pattern applied to using appSettings:

The thing to remember is, these calls are bound to interactions with the applications configuration file and the ConfigurationManager does very little to mitigate the expense for this type of call. As an I/O bound resource, repeated calls can be very costly. Whether you're using appSettings or another config section, the only way to mitigate I/O bound costs is to cache your settings. There are lots of patterns to do this, here is a quick example of just one.

Lets start with an interface representing our configuration settings. Strictly speaking an interface is not required, but makes it easier to inject configurations using the Dependency Injection pattern later on.

Now lets implement the interface. In doing so we need to accomplish the following:

• Load configuration points from file
• Cache the configuration results
• Provide a mechanism to refresh the cached configuration

As you can see this is essentially a singleton pattern. While this may seem a little boring, its clean, testable, and can be extended to provide a wide range of related functionality. The real test is whether or not this is actually any faster... Creating a simple loop test we can get a sense of how this stacks up.  

Here are the results...

As you can see, the difference in performance can be dramatic. Although I used appSettings as the center of this example, the same basic problem exists for any configuration section (Ex. connection strings, custom sections, etc...). Hope this helps.

Tags: performance, design patterns