Modern Discovery: Breaking new silos

First of all, I want to explain why I added a puzzle as an image for this post. This is a metaphor I'm using with several people I work. People sometimes get lost and cannot keep with modern software initiatives. Well, agile is saying that for ages and people still don't get it. There is a huge difference between Simple and Complex systems. More and more making a digital product is from far to be a simple task. However, we still want answers that we cannot get. Sometimes this creates anxiety and we need to learn how to deal with failures, chaos, bad feedback and even that our solution my suck. Ouch thats very dark, right? No, not really, this is the reality, the REAL WORLD is a cruel place and ADULTS need to call tough decisions every day. So what's the deal with simple linear systems? well they imply that quickly you can understand what you are doing and make predictions about the outcomes. Well, digital products are not like that, we cant make predictions on simple linear thinking like A -> B -> C. It does not work like that.

The right Vision for Digital Products

Digital products are not like your uncle old software projects. Why not? well because the world has changed and startup competitors can beat you and from day to night you might lose it all. What's wrong with the project thinking or project midset?

• Focus on the wrong things: Dates, Scope, and Cost.
• Does not take the customer obsession into account.
• Does not use discovery and th way to learn tought experiments. 
• Does not pay technical debts and use technology as a bag of magic tricks.
• It's full of waste: Feature no boddy use, people don't even know it(lack observability).
• It's full of waste: Takes so long to release that people don't care or need to work differently. 
• IF for some reason you got all right, you end up killing your team and need to go to over again.

Why we keep doing this? simple because we don't educate our selves and keep repeating old management dogmas because is that the only thing we know how to do. There are better and honest ways to approach the problem but before we talk about Dates, could we talk about benefit? how we make sure we are adding value to the customer?

As soon as you realize there are at least a million things preventing you to get the date right, such as Technology has changed and you don't know how to do it. The user's changes and you know now them very well, the ways of developing software have changed and you don't master these new ways. User interfaces have changed from desktop to web to text, to voice to XR. Operations have changed to DevOps|SRE. Management has changed and now it does not make sense anymore, no one needs command and control anymore, we need coachs and mentors. The business has changed and we don't need People throwing requirements via word specs into developers. Sales have changed, software distribution has changed, the world has changed. Engineering has changed, are you still talking about relational DBS and governance?

OK if you acknowledge that which is huge, I have some very bad news to give you. Here are 2 super bad news. First could you chance? Are you used to continuous change? If not this will hurt. Even if you make it through here comes the second bad news. All of this things I mentioned before(which are super important) are pointless if we can't generate value. In order to generate value we need to understand the users, have the right technology/architecture and make sure we get the business right these tasks are not easy. In order to get this right, you need discovery it. So it's like a puzzle it will only make sense ater sometime, you cant understand up to the front(and don't need to) and it's not linear you can predict with sticks and carrots.

The Discovery Process

I'm not a Scrum guy. Actually, I believe a good part of the damage was created by scrum. Let's start talking about the role of Product Owner(also known as PO): Let me give some bad news for you. There is not the PO. Yes, the PO does not exist. It cannot exist. Why we cant have the PO role? PO implies you have a simple individual how knows the business, UX and Architecture this might be throught if you are Jeff Bezos or Steve Jobs, not I'm not 100% sure if they got this 3 skills.

The discovery process, needs to happen with triads of people. An architect, a BA and a UX person. Only with these 3 disciplines, we can make good calls about how the product should behave.

The architecture alone is non-sense, I think I don't need to explain that to anyone. However the other 2 alone people think is ok but actually is not ok at all.

BA alone more of times means some bad practices like not doing proper UX and also just getting orders from the business and sending down to the developers, a modern BA is not an Order taker but a leader. The modern BA has no issues with failure, it embraces the failure, a modern BA embrace experiments and is in constant learning about the business, the technology, and everything. A modern BA does not think about UI only, it thinks about other interfaces specially APIs.

Most companies think have the UX alone is a good idea. Thats a terrible idea. UX alone means you will have waterfall projects, so the UX will define all uis and you just need to execute it, ideally in a fixed price, fixed scope project, this is what most companies are doing and is comple waste. So Ux is bullshit? No, not at ALL. I love UX, Ux is super important but it needs to be binding with architecture and BA(who should know about the business).

The discovery process does not write Specs or user stories on the stone. They discovery things and the discovery process sometimes results in a user story other times results in just not asking the engineering thing to code. Yes, and thats one of the greatest results you can have from the discovery team, killing a user story is a good thing, you are saving tones of money doing that.

Why the hell do we need to kill user stories and throw them into the garage? Isn't this waste? Are we throwing money away? Well if you engineer some feature and the users don't like or don't use it well then you are for sure throwing money away because the engineer is the most expensive thing you can have after cloud costs. So you know be sure before throwing anything there.

But I'm 100% sure? Awesome so this means you are not doing innovation. Because there is no sure with innovation, is all about risk, its all about failure and learning from it. Thats why you need experiments and you don't know what will happen.

Breaking Silos

Breaking silos is something beautiful to say however companies are not ready to do it. Why not? because leaders are not ready to do it. Because often means less power and doing the right things are not for the week, only a strong leader can do such changes.

Let's consider 3 classical silos on IT | Product departments. Architecture / Technology is one silo. Ux is nother silo and BA(Business Analyst) is another silo.

Why these 3 silos are bad? Because they introduce delays and create the false illusion that they can work isolated and they simply cannot work in isolation, not in an efficient way. Why not? because no silo has the complete picture, so this silos just make things more difficult. Silos also create the wrong ownership feeling, An architecture my say is not my job to know about UX or Ba might say I don't need to worry about Technology, all wrong;

So we need to tear down silos and have flexible borders where the architect will still be the responsible about technology however will understand the business and know the user so all solutions this team creates are rounded and they don't have loose ends.

BAs often just know about the business needs or strategy, this does not mean is feasible to execute(architecture) or if is nice for the user(UX). Lean Startup and LeanUX do UX is a very agile way, most companies do UX in a super waterfall way. The discovery process make sure developers don't get surprises like? what we should code? are we ready to code this? or is the user really need this and does this UI make sense? The discovery process can and should also kill user stories as the experiments fail and they learn what works and what does not work.

Why you can't get Dates right with Estimates

Every sale, financial and market team need dates. However, get dates right via estimates is almost impossible and become more impossible every year. However we still rely on something that is not reliable at all(estimates). The need for estimates is real. So if we can get them right, could we replace them with something else? I believe thats the right approach. Instead of just saying there is no date we can replicate dates with:

• Business Visibility: Via Kanban boards, Demos and Retrospectives.
• Small 8h user stories: Small stories are easy to manage and predict outcomes. 
• Constant business progress: via talks, experiments, user story mapping, and releases.
• Using OKRs and focusing in quarters rather than anual process helps a lot. 

Even doing all that, people will still want some kind of commitment and dates at the end of the day, which I think are fine and fare if we:

• Have the right architecture and infrastructure in place that allows us to release software fast.
• Have business availability and discuss scopes(MVPs) in order to have smaller releases.
• Run discover the process and produce ~8h user stories(fight variation).

Doing so we can focus on cadence instead of trying to guess several requirements, super up-to-front without UX and Architecture involved. The chance to get this estimates right is lower than getting the first prize in the lotto.

So instead of guessing 400 items in a poor structure, poor defined, poor discovered backlog, lets run the discovery process and use the kanban cadence formula.

Let's say cadence us K. We just need to know how many user stories we need to deliver(discovery process) will be able to get that for us. We also need to make sure storys are no longer than 8h.

Them we get how many weeks we have to the desired date. We just divide the number of stories per the number of weeks and we have the throughput we need per week. This throughput could be called cadence(K).

When we have the cadence we can know how many devs we need just dividing the cadence per 5. Why 5? A single developer works 5 days and delivers 5 (8h stories) per week.

You can also observe our engineering team and tweak things to make sure you can deliver a story per day, You might need some time to tweet your discovery team to produce 8h stories as well but this is much better because you can predict better when you will finish. At the end of the day if a developer doesn't finish a story we are delayed so this is easy to manage.

From Estimates to Cadence 

When you have cadence, you dont need estimates anymore. Actually, you dont want use estimates you want to cadence because if your backlog is breaked properly and you deliver it it will be much better to manage it and have dates in short tern for the business. Dates in long-tern dont work and its impossible to have it. Dates in short tern following this method are perfectly feasible.

There are 3 key elements here: A) discovery process that produces 8h stories, B) Engineering team being able to deliver software with constant cadence. C) Management/business changing a bit how the demand software and break into more continuous deliveries rather than yearly waterfall releases.

TTA (Three Track Agile)

TTA is inspired by Dual Track agile, I just added a third track called foundation. Why do I have a foundation there? Foundation is hilly inspired in Engineering Organizations from Silicon Valley. You can think as a Poor Man solution(in the case you dont have an engineering Organization like in SV). So the foundation focus on Architecture and Technology to speed up the Delivery team like What Observability tools are we using it? Corporate architecture principles like SOA/APIs, Cloud-native, REST, Proper Software Design and much more. This is important to make sure discovery has fewer surprises and can focus on the business. This does not make all technical changed are removed from delivery. But if something could take 4 months to get it, it's not a good idea to discover it in the delivery where you expect to release software every day.  So the TTA it's my personal adaptation between DTA and EO from SVs.  In DTA there is retro-fit for all tracks in TTA is the same deal we just have one more track-focused in the foundation. This foundation could mean different things depending on the size of the product or company, could as simple as a list of technologies and templates and could get as complex and self-service platform solutions like stress testing platforms.  The cool thing is that you can put all these tracks together in a single kanban and have great visibility and the main benefits are:

• Reduce Risk
• Reduce surprises at Development
• Deal with bottlenecks before(increasing the change to get dates right).
• Enabling the delivery team to focus on business value  via constant software delivery
• Reduce waste
• Reduce re-work
• Increase visibility 

I hope you guys liked, I will make more post on this themes soon.

cheers,
Diego Pacheco