The Art of the Queue

Now I start to understand the challenge: This always seemed like it was totally solvable before. "Why don't they just..." But now I get it. I want to manage by dashboard. Have a dashboard that feels like command and control for understanding and making decisions. A common dashboard for everyone to sync on that helps drive meetings objectively and more efficiently. The first problem is that what we need to measure and chart everything is not currently being measured. So bringing my visualization ambitions into reality will take some more work, but I'll get to that in another post. For the rest of this post we will assume that all of the items in the queue have properties necessary for the decisions everyone needs.

Under, Over or between

In my interviews and research a pattern emerged. People often were looking for what is above or below a threshold.

1. Is it ready?
2. Are customers asking for it?
3. Is it within our guidelines and capabilities?
4. Will this result in a sale?

All of these binary questions can be assessed by having a linear rating system for each question, and putting everything on 1 side of it or the other. Is it above or below the threshold?

Sometimes you care about at least 3 groups:

1. Below the minimum
2. Above the maximum
3. In between min and max thresholds

These are still linear, 1d. They are useful for singular views but when you have multiples of them to answer it can be difficult to correlate with each other and understand what the overlapping items are.

Correlating 2 thresholds

Things get a little more interesting when we look to correlate 2 thresholds. First, a couple of ground rules

1. Everything correlated must use the same scale. I'm using 0-1 with double precision.
2. 0 must = bad and 1 = good across everything correlated.
3. The X (first) threshold is more valuable than Y threshold.

If we chart the intersection of 2 thresholds each with only 2 groups I've now re-invented a quadrant chart or 2x2 matrix. It's somewhat validating that I reach a common reference point that everyone understands but this is not very exciting is it? But wait a minute. Why don't we use a "magic quadrant" style chart for queues and thresholds? I found a couple possible explanations:

1. Scalability: You end up with clusters of unreadable items in similar locations beyond a certain scale.
2. Data Density: Borrowing a bit from Edward Tufte a bit on this thought (I took his course and it influenced me since) We end up with a lot of dead space in magic quadrant charts where there are no items. This is fine as a standalone chart, but it make it difficult to correlate with other charts and information.

I developed my own matrix component to deal with these issues.

1. Items aren't clustered, they never overlap. They are placed in a group grid so they can be seen distinctly. What matters most is where they are by the thresholds, the details within can be seen in a drill down. This way you can easily tell the group sizes and proportions as well. I call each threshold combo an Item Group

2. Item Groups have a minimal size, if empty there's just enough to show it's there an empty otherwise it dynamically sizes based on the other groups, this removes all of the dead space that isn't very helpful.

3. This new dynamically sizing matrix needs to work with threshold combinations where either X threshold or Y threshold can be defined with 2 or 3 groups, giving us 2x2, 3x3, 3x2, and 2x3 matrices.

Now this is starting to feel like something tailor suited to the issues.

Scaling to the nth Dimension

This matrix method will only give us combinations of 2 thresholds at a time. Nothing is that simple. I tried adding a 3rd threshold in a 3rd dimension but it was pretty difficult to view.

What I ended up doing was to create layers of these matrices for the various view and a way to view them in aggregate. This starts to appear as a useful way to manage the complexity. The overall aggregate view follows the same rules as the rest in terms of how you read it, but now it represents as many layers of matrices as you want to define and select.

Conclusion

This has taken way too long to come to a simple and flexible solution. I'll discuss more of my practical application of it in other posts. This one is intended to be purely from a data visualization challenge perspective. I developed my components to be more flexible than necessary so I might use them with other queues. If you are interested in applying custom visualizations to your queues feel free to reach out.