Mar 30 2010

SCRUM-tastic Tuesday - Going Dark

Category: Intellectual PursuitsJoeGeeky @ 00:04

In my last SCRUM-tastic post, Manager Interruptus, I picked on managers a little and blamed them for hampering the teams momentum. As I mentioned, it was a little unfair but it served to illustrate something we all know is true... Despite "your" best efforts, constant interruptions can really destroy your ability to be productive. From time-to-time, no matter what you do, you'll find yourself; or your team; in danger or not meeting commitments. When this happens, you generally have four choices... 

  • Cancel the Sprint - If something has happened that is a complete game-changer, then you should approach your Product Owners and discuss canceling and re-planning the Sprint. In Scrum terms this is totally acceptable, although it should be done only when absolutely needed and should not become a regular occurrence 
  • Ask for more time - This may work, although it creates a slippery-slope and could have down-stream impacts that may not be obvious to the development team. Putting aside the issue of breaking the Scrum time-box rules, this could impact things like resource scheduling for other team, which is common in Scrum-of-Scrum scenario's often seen in larger organizations. This could impact business commitments to customers or suppliers, which could have costly legal implications. It could effect marketing campaigns which often have substantial investments and may be time sensitive such as holiday offerings. In some markets licensing restrictions may only give a company a small window of opportunity to get into a market, and delays could cause them to miss the window. In any case, since we burned the managers in the last post they probably won't be keen on this for a while Foot in mouth
  • Let it fail - I covered this in the last post, and this may be an option in some cases, but is generally frowned upon. Assuming it doesn't require super-human effort, striving for success is often better then outright failure
  • Go Dark! - Given the title of this post, I'm sure you guessed we would end up here. This isn't the most ideal solution, but sometimes this is the only "reasonable" option. Lets explore this a little further

When individuals cut themselves off from the team, this could be an indication they're thrashing which is generally a bad thing. In this case, I'm talking about the larger team going dark together for part or all of the Sprint. The goal is to limit the number of interruptions to mitigate the risk of the team not meeting their commitments. Depending on how far the team is behind, you may only need to Go Dark for part of each day or for several days straight. Assuming you can see the danger coming, you should start with the former and only go fully dark if you absolutely need to. The techniques vary widely and really need to be adapted for your team but here are a few suggestions:

Level 1 - Nominate a stand-in

When people start to fall behind you'll often hear complaints about having to attend too many meetings. If this is your case, then you'll need to nominate someone else to attend these meetings and represent the team. The Scrum Master or Team Manager may be ideal candidates for this. Since they are likely aware of your predicament, they should be more then ready to step-up and support the team.

Level 2 - Drop your connections

Whether we like to admit it or not, there are a lot of ambient distractions that tend to draw our eyes away from what we should be focusing on. While it will pain most of us to admit it, our distractions are likely online in one form or another. A few examples include; but are not limited to; the following:

  • Close the Browsers pointing to your favorite social networking site, news/sports cast, and/or blogs (except mine of course). At the risk of stating the obvious... you probably shouldn't be doing this at work anyway
  • Shut down Instant Messengers... It's not enough to set "Away" or "Invisible". To avoid those attention getting beeps, bleeps and honks you'll need to say goodbye to your friends and loved one's, shutdown IM, and get to work
  • Close Outlook, or whatever email client you're using. When you see or hear emails arrive there is a tendency to stop, read, and answer them. In most cases, they can wait a little while
  • Disable Mobile device "Push" Notifications. In some ways, this feature should be an indication we've gone one step too far in being constantly connected. Let go of your Twitter, Stock Alerts, Facebook, and Weather updates for a few hours so you can get something accomplished
  • Etc...

Level 3 - Keep the lines busy

If you work in one of those shops where people just love calling you, then you may need to unplug your desk line, close your VoIP client, and put your mobile phone in Airplane mode. If you're on-call or still need to be reached by your manager, make other arrangements by having them contact the team through a third-party and make sure these details aren't advertised to anyone else. This could be through the Scrum Master, receptionist, or someone else you can trust to hold-back the flood-gates of distraction.   

Level 4 - Leave the building

This may seem a little over the top, but it may be necessary. We've all experienced those moments when noone was around and we got a tremendous amount of work accomplished. In this case, we're just trying to recreate that experience. One of my old teams liked going to the local coffee-shop for a few hours each day. They had fuel (e.g. coffee and scones), wireless internet (often free), most patrons treated it like a library (e.g. quiet), and most important they had no drive-by interruptions from colleague's. There are many variations to this technique. Assuming you trust your team, they could work from home from time-to-time. You could book a conference room for a long period of time and all work from laptops. In most cases, people will typically avoid bothering people who appear to be in a meeting. Your company may even have other facilities you can use.

At the moment, you might be trying to figure out what you can get away with where you work. Whatever your case, you'll need the support of your Scrum Master and Team Manager. Ideally, they'll run interference for the team to make success as achievable as possible. Even if you disagree with my specific examples, hopefully you can see how techniques like this can be useful in varying degree's. Although I implied this is needed to avoid interruptions from people, it is worth pointing out this can be useful for those high-value last-minute business "requests" that need quick solutions. In any case, give it a think, adapt to your team, and hopefully you'll make it over that hump...


Mar 26 2010

Structuring my storage

Category: Intellectual PursuitsJoeGeeky @ 00:04

In the late 90's, a term was born which spawned a whole range of non-relational storage facilities. What is this term? NoSql

NoSQL is a movement promoting a loosely defined class of non-relational data stores that break with a long history of relational databases and ACID guarantees.
- Wikipedia

There are a wide range of purpose-built solutions out there ranging from document storage systems to Tuple Space data grids. Each of these targets a specific niche sacrificing more traditional SQL-like patterns for other advantages (Ex. speed, portability, structure, etc...). Even with so many differences between them, these architectures generally share a number of common characteristics and while they may sounds a lot like traditional databases, their implementations can be quite different. Consider the following common Structured Storage components:

  • Store - This is the storage medium. This is commonly a file (or many files). However, in this modern distributed world of ours this can be virtualized in many different ways
  • Index - This can be one or many different representations of part or all of the stored data. This pattern is generally used to optimize search or data location routines. Any one store can have many different Indexes to facilitate its needs
  • Cursor - This is used to iterate through the store generally pointing at one record at a time. This is similar to an Enumerator or an Iterator although one store could have multiple cursors at any one time. The cursor is often the point from which data is written to the store or read from it

Understanding these basic principles can make it easy(ier) to create your own purpose-built store for meeting any specific needs you might have. I recently built a custom point-to-point queue and needed it to be durable. In this case, I wrote a store to guarantee delivery of queued messages on both the sending and receiving ends of the queue. In doing so, I was reminded of a few valuable lessons:

  • Technical Debt - To make a custom store suitable for highly available and/or performant applications, you will need to employ a number of advanced techniques. This includes asynchronous and possibly distributed processing, replication strategies for fail-over, etc... These issues are not trivial, and can require large investments in time and money to make them work correctly. If you have these needs then it may be better to go with an established technology
  • Disk thrash - It may not seem obvious, but high-performance persistence technologies need to be aware of what a storage medium; such as as disk; can and cannot do well. Think about how disk heads move. If your cursors, data readers, and/or data writers behave in a manner that would cause the disk heads to jump around, you will lose precious time just due to the mechanical limitations of the disk. Do a little research and you'll find patterns to help mitigate this kind of performance hit
  • Describe your data - When you're architecting your store, keep in mind that you need to store meta-data along with the data you intend to store. This could include data used to generate metrics, versioning, structure details, arbitrary headers, or whatever. While it may cost more, make sure you give yourself room to grow 
  • Familiarity - Take a sampling of developers and show them a database, tables, sprocs, etc... The vast majority will know exactly what to do if change is needed. Compare that to showing the same developers a proprietary storage solution. While they may be able to figure it out, it will take a great deal more time and energy to make change, isolate bugs, etc. Like it or not, most of us recognize the classic database model. Having something people recognize can be worth a lot, so don't underestimate the value of older patterns  

In today's complex environments, experience with the aforementioned patterns can really come in handy. Investing a little energy in this area can be worth it, even if it is just done as an intellectual pursuit. Just remember, this is all about purpose-built stores. Don't feel like you need to copy or replicate functions from other tool sets. If you are, then maybe you should just use those tools... Wink

You're welcome to take a look at an early version of one of my stores. Although its not the best example, it met my needs. Here is some SmellyStorage.

Tags: , , ,

Mar 23 2010

SCRUM-tastic Tuesday - Manager Interruptus

Category: Intellectual PursuitsJoeGeeky @ 01:12

In the last couple of posts, Dealing with naysayers and Big projects and Epic tasking, I focused on the team members that seem to love complaining. Unfortunately, this isn't completely fair to those that have bought-in to the process and are just trying to meet their commitments. For them, let's turn the discussion to a type of impediment that can make any process or committed team fail... Managers... or at least some of them. Again, this may not be fair, but let's explore this a little more and I think you'll get my point.

For the sake of discussion, let's just assume everyone is bought into the process. This buy-in assumes that everyone knows that one of the key process tenets in Scrum is that once the Team has made a commitment, they need to be left to meet that commitment. Unfortunately, managers are vulnerable to acute cases of amnesia which often manifests shortly after the Sprint Planning session. Whether or not this is truly a medical condition, it can have detrimental effects on the team's ability to stay focuses and maintain momentum. Treatments for this affliction can require a delicate touch and have mixed results, but before we discuss treatments let's look at some of the symptoms and explain how they can affect the Team. 

Multi-Tasking, Context Switching, Task Switching

Some would say the ability to multi-task is a crucial skill for any IT professional. The problem is... us humans just don't do it very well, although many of us think we can. Do a little Googling and read what the productivity experts have to say. Their agreement on the matter is almost unanimous... we often become less productive when we try to multi-task. Managers need to be aware of this more than anyone since they're the first line of defence. Most teams will have some ability to absorb unplanned tasking; which is often represented in the team's productivity rate; but managers need to make sure people don't take advantage of this "potential" capacity.   

Here's the main problem... when someone is working on a task and they're interrupted, it takes them considerably more time to regain the same momentum they had before they were interrupted. This includes drive-by questions, phone calls, email, visits from the manager, and meetings, meetings, meetings. Depending on the task, this can be really costly if the developer has to regain complex or nuanced information before they can restart their work. Some interruptions are inevitable, but constant interruptions can put the team at risk of not meeting their "real" commitments. The manager needs to be aware of the amount of context switching occurring in the team; especially when it's their fault; and work with the Scrum Master to deal with excessive interruptions. Some might say the interruptions need to be "managed", but that's just me trying to be cheeky. 

One reason this might be the managers fault, is based on the typical manager/employee dynamics surrounding delegation. This is fine to a point, but if the managers aren't careful they can delegate the team right into failure. We can assume this isn't conscious neglect. Maybe they're naively oblivious to the damage left in the wake of them trying to be "good" managers... What?... It could happen!... At some point, managers; and other supporting staff; need to become self-sufficient and carry the load without the Team.

Let's just add this "one" more "little" thing...

If your Sprints are the average three week length, there's a tendency to think that with so much time we can add just "one" more "little" thing. This is another famous manager line, but it's a slippery-slope. If the manager wasn't afflicted with 'the illness', they would realize they're asking for trouble; or at the very least, adding risk. The reality is, there are very few "little" things that can be added without having a larger knock-on effect. New stuff; whatever the size; needs to be designed, coded, integrated, tested, documented, etc... Before you know it, the dev has lost valuable time on something that wasn't originally a commitment. As a rule, there should be no new or altered commitments once the Sprint has started. 

Possible Treatments

Communicate the problem - The best thing to do is have a team meeting with your manager(s) and tell him/her what the concerns are. The Retrospective Meeting is a great place to start this discussion. Depending on your relationship with your manager this may or may work, and you may need a more subtle approach. In this case, try reducing your teams productivity rate and subsequently reduce your commitment at the next Planning session. If you keep doing this, your manager is bound to ask what is going on. This will give you another opportunity to articulate how interruptions are endangering your team's ability to meet their commitments. To be effective in making your point, you'll need to have some evidence. As they happen, try adding unplanned/uncommitted tasks to whatever tracking system you're using. This will give you a valuable point of reference and may also reflect on the burn-down chart which can help reinforce your point.

Learn to say "No" without saying "No" - This can be hard in some shops, but may be necessary to prevent the teams failure. Since "No" can be received negatively, try approaching this from another angle. This is a bit Freudian, but it works.

  • Start by suggesting the new task be a "priority" for the next Sprint. This sounds like "Yes" but implies a "No" for the current Sprint without saying it 
  • Say "Yes" with the Proviso the manager identify what "commitmentshould not be done as a result of adding the new task 

Batch your interruptions - As valuable resources, it makes sense that some questions can only be answered by the Team. If you're suffering from frequent drive-by questions. Ask your manager to save them up and ask more questions with fewer interruptions. Maybe he/she can agree to meet the team once a day or every couple days to have a time-boxed Q/A session. For this, I recommend following stand-up rules so things don't take too long. 

Let it fail - No one likes to fail, and I'm not suggesting sabotage. With that said, there's no sense fighting the inevitable. Everyone has to be made aware of the consequence of not keeping the larger commitment to the team. Some Sprints can require extra-ordinary effort, but they can't be planned with the expectation of Herculean efforts on the part of the development team. That's a recipe for disaster. 

This isn't meant to represent a complete list of treatment options, but may give you a few ideas on how to handle this. Whatever you do, try and remember that everyone has their own perspective, and odds are good they're busy as well. In most cases, the problem(s) comes down to communication, or the lack of it. Start with the direct approach, and resort to Freudian manipulation only if you need to. 

I picked on managers a little bit, but the truth is, they're a valuable part of any team being successful. With that said, we can all slip into bad habits and sometimes need to do a little Freudian introspection to reflect on our own behavior. For full disclosure... I've done my share of managing, and I've certainly been guilty of doing things mentioned above. Like anything else, the Team and their manager(s) need to find a balance to ensure the larger business objectives can be met without killing your team in the process (pun intended). 

Before wrapping things up, there is one more thing... What about the Scrum Master? ... Great question!... Since they facilitate every daily Scrum and should be the first to hear about impediments, they should be well aware of what kind of stress the Team is under. Consequently, they should be the first one to tell the manager when its time to back-away and start helping keep outside interruptions from the Team. So get to work you Scrum Masters... Wink


Mar 19 2010

Putting your DataTable on a diet

Category: Tips and TricksJoeGeeky @ 01:08

As the name suggests, the System.Data.DataTable is a fat object type. It's true that there's a lot you can do with this class, but this may come at a pretty high price. To demonstrate this, lets start by creating two in-memory structures and compare their sizes using WinDbg. The size of each structure will be a conservative 500,000 records, each with a puny 7 columns of 32 Bit Integers. If you've developed high-performance applications you know this is a pretty conservative amount of data, but this should still make the point.

const int MaxItemCount = 500000;

Our first structure is built around the suppossed fat DataTable. 

var table = new DataTable("MyFatTable");
table.Columns.AddRange(new[] {new DataColumn("A", typeof(int)),
                                new DataColumn("B", typeof(int)),
                                new DataColumn("C", typeof(int)),
                                new DataColumn("D", typeof(int)),
                                new DataColumn("E", typeof(int)),
                                new DataColumn("F", typeof(int)),
                                new DataColumn("G", typeof(int))});
for (int i = 0; i < MaxItemCount; i++)
    DataRow row = table.NewRow();
    row["A"] = i;
    row["B"] = i;
    row["C"] = i;
    row["D"] = i;
    row["E"] = i;
    row["F"] = i;
    row["G"] = i;

Next we'll create a more traditional object collection so we can see what the DataTable "might" be costing us.

public class DataItem
    public int A { get; set; }
    public int B { get; set; }
    public int C { get; set; }
    public int D { get; set; }
    public int E { get; set; }
    public int F { get; set; }
    public int G { get; set; }

IList<DataItem> data = new List<DataItem>();
for (int i = 0; i < MaxItemCount; i++)
    var dataItem = new DataItem();
    dataItem.A = i;
    dataItem.B = i;
    dataItem.C = i;
    dataItem.D = i;
    dataItem.E = i;
    dataItem.F = i;
    dataItem.G = i;

If we run the sample and create a Dump file, we can see how much memory is being consumed by each type of structure. The DataRows in the DataTable make up almost twice the amount of memory as that of the more traditional object collection (e.g. DataItems). Keep in mind this was a really simplistic sample, and real-world structures will likely be much larger (in row count and columns). Aside from memory pressure, the DataTable will cause your application to spend more time in Garbage Collection, which will translate to your application being slower.

Spend a little time profiling your application and you'll likely find other large concrete implementations with hidden costs. In this case, the solution is easier than you might think. In many cases, all we really need from a DataTable is the underlying DataReader. Assuming that is true in your case, all you really need to do is implement a custom System.Data.IDataReader. This is a fairly large interface but it is really easy to work with and many solutions don't actually need every method implemented. This allows you to start simple and add additional features when and if you need them.

using System;
using System.Collections.Generic;
using System.Data;
using System.Reflection;

public sealed class GenericListDataReader<T> : IDataReader
    private readonly IEnumerator<T> _enumerator;
    private readonly List<PropertyInfo> _properties = new List<PropertyInfo>();

    public GenericListDataReader(IEnumerable<T> list)
        _enumerator = list.GetEnumerator();

        const BindingFlags flags = BindingFlags.Public | BindingFlags.Instance | BindingFlags.GetProperty;

        /* Map properties to columns based on the order in which they occur */
        foreach (PropertyInfo property in typeof(T).GetProperties(flags))
            /* Only support adds properties that can be mapped to database types */
            if (property.PropertyType.IsPrimitive ||
                property.PropertyType == typeof(string) ||
                property.PropertyType == typeof(DateTime))

    public bool Read()
        return _enumerator.MoveNext();

    public int FieldCount
        get { return _properties.Count; }

    public string GetName(int i)
        return _properties[i].Name;

    public Type GetFieldType(int i)
        return _properties[i].PropertyType;

    public object GetValue(int i)
        return _properties[i].GetValue(_enumerator.Current, null);

    public void Close()

    public void Dispose()

    #region IDataRecord Members
        /* throw NotSupportedException for all remaining members */

    #region IDataReader Members
        /* throw NotSupportedException for all remaining members */


Get the full class GenericListDataReader.cs (4.30 kb).

This is a really basic wrapper around enumerable types. As a side benefit your Data Reader is inherently Mockable which makes testing easy as well. As an IDataReader, this structure can be used to bridge the gap between traditional object structures and data persistence layers. Here is a simple example using the SqlBulkCopy structure to persist our previous collection into a database.  

using (var dataReader = new GenericListDataReader<DataItem>(data))
    using (var bulkCopy = new SqlBulkCopy("A Connection String"))
        bulkCopy.DestinationTableName = "Target Table Name";

        bulkCopy.ColumnMappings.Add(0, "ColumnA");
        bulkCopy.ColumnMappings.Add(1, "ColumnB");


So whats the lesson here? It has two parts... As I said in the beginning, the DataTable is fat. Also, if you're writing high performance applications, look closely at those "free" concrete implementations, they may be costing you more than you think.

Tags: ,

Mar 16 2010

SCRUM-tastic Tuesday - Big projects and Epic tasking

Category: Intellectual PursuitsJoeGeeky @ 01:39

In my last Scrum article; SCRUM-tastic Tuesday - Dealing with naysayers; I talked about common complaints heard when trying to employ the Scrum process. As I mentioned, those types of comments are not really unique to Scrum and are often heard when trying to implement other processes by those that don't feel they need any process. There was one question that I purposely left out which requires some dedicated focus.

This work is too large to fit in Scrum.

This brings up a couple separate issues... "push-back" and managing individual backlog items that are spread across many Sprints. The push-back is easily dealt with, by using a little common sense. The naysayer is attempting to suggest that large projects cannot be accomplished using iterative or Agile processes. Assuming they can say that with a straight face, any experienced manager should see right through it. Just take a look at any Gantt Chart for a typical waterfall process and the non-sense should become evident. Even if the decision to use Scrum vs. some other process is open for debate, this line of thinking is a slippery slope and often leads to cowboy-development initiatives which can easily fail, fall behind schedule, exceed budgets, or miss customer expectations entirely.

At the risk of sounding redundant, Scrum strives to have its participants set a short-term goal (e.g. the commitment), allow it to happen, and then retrospectively assess what they have done. This frequent repetition allows its participants to make minor; even major; course corrections based on their experiences throughout the life of a project. But what about larger architectural considerations? Good question... Just because you're working in short iterations does not mean you can't work from a larger architectural plan. Adherence to this plan is left to the Product Owners (or architects) to ensure backlog items are prioritized appropriately before each planning session.

If you're working on a big project you may find that you have backlog items that are either so large they can't be accomplished in one iteration, or represent work that applies to every Sprint. These situations are actually very common in Scrum and have many names (e.g. Epic's, Saga's, Enterprise Stories, etc.). Requirements surrounding compliance with specifications are a common example of this. If you've developed e-commerce sites you may have had to comply with things like PCI, Section 508, OWASP, and others. There is a tendency to make these types of things backlog items although I think this may be a mistake. Generally speaking, backlog items should represent "what" the customer wants. These types of requirements are really constraints on "how" it will be implemented. This may seem like splitting hairs but there is a reason for this line of thinking.

The problem (or at least concern) is backlog overload or making the "process" become the product rather than focusing on prominent features. If Product Owners keep presenting the same backlog items over and over again, these items can take valuable time away from consideration of more substantial feature-sets during planning sessions. This can start to turn into white noise, and pretty soon no one is really paying attention. I'm not suggesting specifications aren't important, I just think they should be planned and dealt with in a different manner. I like to think of them as "constraints" rather then backlog items. These are things that shape "how" the developer will implement "what" the customer needs. Constraints are a natural part of defining "what" is needed, so it shouldn't seem strange to add them to a given User Story.

This is a bit Freudian, but the goal here is to keep the main focus on the larger User Story as apposed to any one constraint and the smaller the number of stories in planning the more effective your meetings will be. To make up for any concerns related to not making specs a first class backlog item, try making your specifications a focus-area in your test plans. This gives you visibility into overall compliance and will be a continued reminder for everyone involved.

No matter how you decide to manage such requirements, these concerns are not really unique to Scrum, or any other Agile process for that matter. With that said, the Scrum process does provide a framework which may just make it easier to manage within your team. If nothing else, these considerations may just help you do a little push-back of your own on those naysayers.


Mar 12 2010

A faster way to take out the Garbage

Category: Tips and TricksJoeGeeky @ 02:15

If you've been working with Intermediate Languages for any amount of time you're probably aware of how expensive Garbage Collection can be. This invariably leads to discussions related to how to properly Dispose objects; which is very important; or how to "force" collection which is generally a really bad idea. I do not intend to revisit these overly blogged topics, and will assume you already understand the benefits and/or consequences of both. With that said, there is one aspect of this discussion which is not widely understood within the .NET community. Unfortunetaly, this can represent a missed opportunity for many applications suffering from collection performance issues.

Listen closely to your collegues when discussing this subject and you will likely hear them speaking of the "the" Garbage Collector (GC), and herein lies the problem. There isn't one GC, there are actually three different GC configurations, each having unique characteristics and potentially some drawbacks for certain types of applications. Lets take a look at each one and then we can see how to override the default selection made by the .NET Framework.

Workstation (Non-Concurrent)

This GC has one heap per process and performs all collection on the applications main thread. Having multiple processors and/or cores does not change this configuration. Although not the default for any application type, it can have subtle advantages for non-graphical applications such as Consoles.

Workstation (Concurrent)

This GC has one heap per process and it has one dedicated GC thread per process. Having multiple processors and/or cores does not change this configuration. This GC is the default for Console and WinForm applications. The dedicated GC thread allows the application to remain more responsive while performing collection. For WinForm applications, this can be more pronounced as a result of windowing refresh/invalidation. 


This GC has one GC heap per Processor and one dedicated GC thread per GC heap. As indicated by its name this GC is optimized for server-based applications. This provides better scalability by allowing a process to create objects in multiple heaps. This allows for better load-balancing and consequently is the default for all ASP.NET applications, including web services.

As it turns out, the Server characteristics can be really beneficial for other Server-style applications like Windows Services. Enabling this GC can be quite handy but the question is, how to know when this may be better for your application. Unfortunetally, the answer to this can be quite varied but there is one simple performance counter that may be a good signal to enable this capability. Take a look at the following Performance Counter:

This counter tells you how much time; as a percentage; your application is spending in GC (duh). If your application is averaging percentages above 20%-30% then enabling the Server GC may be beneficial for you. Enabling this or other GC modes can be done in the config file as follows:

    <gcServer enabled="true"/>

As with anything, you need to test, test, test to ensure this provides a benefit to you. Enjoy...


Mar 8 2010

SCRUM-tastic Tuesday - Dealing with naysayers

Category: Intellectual PursuitsJoeGeeky @ 18:51

No matter what process you try to implement there is always one constant... "Naysayers"...  What is a naysayer?

One who frequently engages in excessive complaining, negative banter and/or a genuinely poor and downbeat attitude. ... They have the capacity to rant and whine for hours on end about the most insignificant inconveniences.
- Urban Dictionary (

We've all heard the complaints and they generally follow a common theme:

This process won't work for "our" kind of work.

This is a common misunderstanding. Aside from Scrum largely being a development process it is not prescriptive with respect to the type of work being undertaken. This process can be applied to graphic design work, engineering work, testing, product releases, defect resolution, etc. The process makes no mention of any one technology stack, operating system, application type, style of development (e.g. peer-programming or individual developer), etc...

This kind of task can't be estimated.

This is another common misunderstanding. Scrum does not require an "estimate", but it does require you project a time-box around everything you are setting out to accomplish (e.g. the commitment). This helps manage momentum and prevent failures from spinning out of control. More importantly, this helps prevent things from going unseen by those that have to manage the project and manage customer expectations. Essentially, you set out to accomplish something over a given period of time, then come back together at the end and reassess. Scrum is an iterative process and repeats as many times as needed to deliver the required results. 

There is too much paperwork. I'm a developer, not a tech writer.

There are two problems with this... First, Scrum is a very lightweight process with respect to artifacts and has no specific requirements surrounding the fidelity of artifacts. If you have really complicated or overly detailed artifacts you can't blame the process, blame yourselves or your company. I have witnessed many shops managing their work with a wall of sticky notes and that's as simple as documentation gets. Second, developers who think they never have to write anything down are kidding themselves. Putting the obvious design and requirements artifacts aside, in the real world businesses need to be able to capture certain aspects of product development in writing to protect and maintain their investments. 

This process slows us down. We could work faster without it.

I love this one. This is the notion that if the management teams would just leave the developers alone, developers would be more efficient. Truth is, working in total isolation seldom leads to any real success. Putting that aside, the Scrum process demands very little from its participants in terms of time. Besides the daily stand-up meeting; which is only 15 minutes long; the only required meetings are the Planning Session, Demo/Review, and Retrospective, and these only occur about once a month. I can't think of too many professions that can really claim so few meetings. Heck, I had more meetings in a week working as a kitchen slave in France.

There are plenty of others, but you get the point. So when this happens what can you do? Unfortunately, there is no "one" answer but there are a number of patterns that can go a long ways to helping:

  • Be consistent
    • Once the team has decided on how it will implement its process, do not allow deviation. Do not sacrifice schedules or milestones. The team will need time to settle in and see the forest through the trees. In the mean time, you need to keep them on track... even if it hurts. Use the retrospectives at the end of each Sprint to introduce changes over time
  • Be pedantic
    • Don't sacrifice the details. This may not make you any friends but you have to be the details police. Avoiding this creates a slippery slope and before you know it, no one will be following the process
  • Get unequivocal support from the management team 
    • Although this can be tough for some, you can only be successful if your management team is totally committed. They need to be stead-fast in their commitment to those charged with implementing the process. If there are concerns, take it offline, and do not argue in front of the team
  • Let them make the process their own, but don't compromise on the tenets
    • Every team has their own quirks, and may have ideas that will work very well for them. Assuming these ideas do not sacrifice the basic process tenets, let them choose their own details. If they feel a sense of ownership, adoption will be much easier. With that said, they need to understand that these ideas will be subject to all the aforementioned points

Well, thats it... Hope this helps.


Mar 5 2010

When "Activator" is just to slow

Category: Tips and TricksJoeGeeky @ 05:00

Here's another quick tip; again... pun intended. If you are writing high performance dynamic code you have likely found yourself using the "Activator" to instantiate instances of objects. This is commonly the case when employing Inversion of Control (IoC) patterns, or using reflection to discover and create types at runtime. Here are a couple typical examples:

IFoo foo = (IFoo)Activator.CreateInstance(typeof(Foo));

IFoo foo = Activator.CreateInstance<Foo>();

These approaches can be perfectly reasonable until you are constructing the same types often (as with IoC) or you have more substantial performance requirements. So what's wrong with using the "Activator"? In a couple words... It's slow! Don't take my word for it. Build your own test rig and it should become obvious.

Still not sure? What you need is another point of reference, such as a more performant alternative. Here comes Lambda Expressions to the rescue. Lets see how we can leverage this capability to rival the Activator.

private static Func<T> GetConstructor<T>() where T : class, new()
    ConstructorInfo constructorInfo = typeof(T).GetConstructor(new Type[0]);
    return Expression.Lambda<Func<T>>(Expression.New(constructorInfo))

The above routine will create a compiled constructor for a given Type. Although you have to keep track of this state, it can easily be managed within a Type construction Factory such as those built into IoC Containers. With this method you can create as many instances as you need and then release the associated resources.

var constructor = GetConstructor<Foo>();
IFoo foo = constructor();

If we apply the same test rig used previously and retest with this approach, it is easy to see how things stack up.

Although not covered here, this sample can easily be extended to support parameterized constructors. Enjoy, and use in good health.


Mar 2 2010

SCRUM-tastic Tuesday - Maintaining momentum and combating burnout

Category: Intellectual PursuitsJoeGeeky @ 03:26

If you've worked in a Scrum shop for any amount of time then you already know that momentum is the name of the game. If Scrum had a mantra it would likely be something like... "Just get on with it...". This can often translate to a sense of urgency, and depending on your perspective, it can be a good thing or a bad thing. In many ways, this straight-forward mantra is what attracts some people to this process, although without proper management you can quickly lose momentum and burn people out. Don't believe me? Google it, if yee doubt the claim.

Do a little reading and you will quickly find the consequences of these conditions can lead to lost productivity, reduced quality, and more often then not valuable people quiting. To make matters worse, techniques to combat these conditions can be quite varied, depending largely on the personality of your team and/or individual team members. Putting that aside, lets try and enumerate some practices that can help mitigate some of the aforementioned concerns.

Intellectual Spiking

In my experience, developers tend to be some combination of artist, scientist (sometimes mad), inventor, and for some... masochist. While we can't cater to the last one we can do something about the former items. If you think about it, there are always things the team wish they could work on. This could be new product ideas, technology exploration, graphic artwork, animations, design patterns, etc. Assuming these projects are not too far off the track, side projects can go a long way to giving developers something to look forward to. If you do a little research it is easy to find plenty of examples of how these types of projects can often lead to huge successes for companies, so this is not all that crazy.  

Invest in Automation

This section could easily be called "Repetition is Evil". Monotonous tasks will absolutely drive devs crazy. While this could be an entire article in itself (note to self), the message here is simple... get rid of the drudgery. This not only makes people happier, it can lead to huge cost savings for companies by letting devs work on more productive tasks, and can lead to huge increases in quality and capacity. Here we are talking about letting teams adopt new tools, build Continuous Integration infrastructure, testing rigs, deployment routines, etc. Some companies may wince at the thought of shelling out a couple grand on new tools, books, or whatever, but anyone who knows anything about this business will tell you it is usually worth it.

Mix up the Roles

This can be tough in some organizations but can really produce amazing results when viewed from a teaming perspective. In this scenario, we are taking someone out of the normal commitment pool which can be a bit of a break by itself. Instead, we can rotate the dev through a variety of other roles. This can be used to help people see each others position from a different perspective which can lead to a better appreciation of what each role provides to the others. Here are a few examples:

  • Change places with the Scrum Master: It can be tough running between teams, organizing your own team, dealing with conflict resolution, facilitating meetings, making on-the-spot decisions, and meeting the individual demands of each and every team member. The Scrum Master can be the unsung hero and everyone should spend some time in their shoes
  • Become one of your users: This is a great shadowing technique... Send some time in your users shoes and you will usually walk away with a better sense of what works and what doesn't. This can be a great motivator for improving products, enhancing usability, and keeping a dialog between the devs and end-users
  • Be the support monkey: Most teams have some kind of support burden which is usually impossible to estimate. Take one person out of rotation and let them answer these calls. This can relieve the pressure for others and provide everyone with much needed visibility into things they may not have written themselves. Between calls, they can start picking away at open or lingering bugs, or take care of other housekeeping issues 
  • Become the enemy: If you have products that are really concerned about security... Spend some time trying to break into your own system. This can be eye-opening and fun

Rethink your productivity rates

In a previous article (SCRUM-tastic Tuesday - Estimation guesswork) I discussed the notion of a productivity rates. This is the number of hours per day each team member is expected to be making progress towards the completion of committed tasks. This tries to acknowledge time spent on the phone, answering emails, attending meetings, training, lunch, etc... No estimate should extend beyond this number. If people are consistently frazzled by the end of the Sprint, then you may need to rethink your productivity rate to give people a little more breathing room. If everyone is always rushed, defects and/or implementation compromises will start to develop which; in the long-run; can be very costly.

Embrace your pain-points

Each team knows what they are, and usually know how they would address them "if" they were given the time. Let them do it. If something is holding your team back then it makes sense to let them deal with it.

Share the "New" work

There is just something pleasant about creating your own project from scratch. While you may be constrained by team or organizational standards, you are not constrained by having to reverse engineer a lot of Intellectual/Technical Debt. When new projects arise or old projects lead to creating a new set of libraries, solutions, etc. share the wealth.

Don't forget play time

I have never been part of a dev shop that only worked 9 to 5. Developers work long hard hours, and they need a break just as much as anyone else. There are lots of ways to support this, and companies that understand this will usually get a better return on their investment. Ideally, you want to give developers time away from their desks but engaged in something together. Depending on your corporate and/or team culture options can vary widely. Here are a few ideas...

  • Send the team to a nerdy conference like TechEd, Code Camps, or designer events
  • Send the team out for a night on the town. This could include, dinner, concert tickets, and maybe even drinks
  • How about team paint-balling... Let them get rid of some of that aggression. This works best when they can challenge the management team
  • Sponsor a game night. This could be a giant Halo event using corporate projectors, a few bean-bags, poker, Uno, or whatever. Buy a few drinks, pizza, and taxi-fare and let them have some fun

Hopefully, this provokes some thought about how you can manage some very costly concerns. Trying weaving some of these ideas into some of your Sprints and you may just have a happier team in the end. As a side note, the issues presented above plague many Agile processes within the development community so feel free to adapt your reading accordingly. With that said, this article is titled "SCRUM-tastic".