by Russ Miles, Wednesday 10th February, 2016

I'm rapidly pulling together the code for Book 2 of "Antifragile Software: Building Adaptable Software with Microservices".

As this extends into a number of repositories, both staging sandboxes for experiments as well as more static examples lifted directly for the book, I thought it was time to collect the lot together into a organisation on GitHub that readers can subscribe to and follow.

The Antifragile Software GitHub organisation is born. If you're reading the book and want an early glimpse of the supporting code as I bring it together, this is one place to start. Also, if you haven't already, follow me on twitter as I'll be announcing small updates to the book there (rather than drowning the readers in day-to-day email announcements!).

I was just introduced to this wonderful talk by Richard Cook on systems resilience by my good friend, Sylvain Hellegouarch

Richard's main point seems to be that the fundamental battle any software delivery and operation team faces is made of three, often implicit, boundaries:

• Responding to economic need to avoid failure: Adding or updating features, supporting different expectations...
• Coping with workload: Teams need to feel confident and at ease they can deliver. If they feel overloaded, they either collapse or go to sleep mode
• Enduring the stress of reality in time: Will the software support the load? Will it fail? What will happen then? Can we mitigate against this? The problem lies in the view that systems can be planned and control from the initial pool of information, a la the Plato story from my own talks! This has led to waterfall methodologies - research and plan enough so your software will be formally designed.

In the early 2000s, it was acknowledged that the initial view was only a partial understanding of the system’s input and so agile methodologies were gradually defined. The aim here was to iterate as often as you can so as to correct your path early.

Although a foot in the direction, this did not pan out as well as one could have hoped for. Indeed, changes could be introduced during the life of the project, but they still could take a long time to be delivered into production. This was the Elephant in the Standup effect; where your software becomes grossly in conflict with reality over time.
​
It's time for this to change. Right now the process can absorb incoming changes but the software, something you might hope would be wonderfully malleable, resists these changes.

If we don't architect and design to thrive on change then we are destined to build systems that will be perceved to be holding the business back.

We need to fix the software design and architecture, not just the process. This is at the heart of Antifragile Software as enabled by microservices.

While enjoying the crazily good beers at Aufsturz with my good friends and colleagues Christian Dupuis and James Carnegie we couldn’t help but have a quick chat about a new term that we’d heard at the conference from, we think originally, our friend Chris Richardson:

"Heritage"

But I have a feeling that as the term was mentioned at the Berlin Microservices conference, at which i was speaking, the term wasn’t being used to refer to a beautiful, third-of-a-ton motorcycle Heritage Softail...

More likely in this case it was meant to mean:

“Heritage Software"

I am a big fan of this new term. It's much better than the term it replaces, which is the rather negative 'Legacy Software'.

Legacy feels like something we could leave in a landfill somewhere and forget about. Legacy software feels like something to be ashamed of. Something to be avoided, not something to be admitted to.

Legacy software has so much baggage tied to it that it feels like the enemy!

But the truth is we create legacy software all the time! As I said on stage at this very event, "Legacy is everything you created before lunch". The danger is that if we treat our legacy with the disdain that the negative term implies, we're only contributing to the problem. The best developers I know just don't think of legacy in such negative terms, and that's why a new term might be useful.

That term could be "Heritage Software".

It’s your heritage, it’s your responsibility, it’s your reputation, and it’s going to follow you.

It’s also much more positive, freeing us from the negative, ugly reputation that "Legacy" had forced upon it. And seeing as we are, at all times, creating this 'heritage' then treating it with more respect, responsibility and even perhaps fondness could be a very good thing.

Imagine looking at your software as something to be proud of because it's your "heritage"!

So next time you're working on some else's 'legacy' try to think of it as a 'heritage', it might just make you kinder towards it and, after all, you'll be hopefully leaving behind your own heritage to be proud of too.

I get asked all the time "Why Microservices?", mainly by lovely people who have got a little caught up in the term itself. The question is often followed with something size-related such as "How big should my microservices be?".

This is not wrong! After all, 'micro' is in the name ... unfortunately.

The answer though is a little more complicated, as explained in our Antifragile Software book. The main reason I've seen for people to adopt microservices is that:

"The business needs to adapt and we need our software to keep up!"

As agility has been introduced and refined into the business the bottleneck often shifts to the software itself. I call this the "Elephant in the Standup" in my talks; the elephant is all the software you have that is now going to be in conflict with today's changes.

​Software is essential to the modern business and you only have to check out the excellent Phoenix Project book to get a wonderful explanation of how software likely is your business. Unfortunately that means if your software is slow to react to the increasingly rapid change in business need, then it is the software that becomes the inertia in the system.

Genuine complaints from the business usually take the form of:

• lack of flexibility
• poor performance 
• incapacity to deliver fast enough
• loss of confidence in the development teams...

In other words, software development is perceived as letting down the business.

This is why we need adaptable software, and microservices offer a great opportunity to enable it.

At a basic level, most business leaders would agree that they want to:

● Address new, and potentially not yet anticipated, opportunity
● Listen and learn from their market and customers
● Re-align the products themselves, quickly, to these learnings

In short, the modern business needs to adapt on a continuous and relatively unplanned basis. When a business cannot adapt fast enough, it fails to address those goals and opportunities.

By designing software to thrive on change will the software delivery be able to gain its true value as partner in success of the company.

Software that thrives on change is Antifragile Software. Antifragile Software through Microservices is our mission here at Russ Miles & Associates.

I was going through the AWS Lambda main presentation page yesterday and I started to wonder if it would be possible to create your own stack with the same feature set.

First off, AWS Lambda is a stunning programming model. A cross between stateless distributed functional programming at scale (say that fast ten times if you dare) coupled with zero-administration requirements. In other words, you write a function and AWS Lambda runs it at the appropriate scale for you.

What’s more is that, AWS Lambda can react to events following the event sourcing pattern. So you don’t keep any internal state, AWS Lambda executes your function with the event data.

​Our job, as software developer, has just become almost too simple. The good news is that we can now focus on the value we are aiming for rather than the plumbing and machinery underneath. A whole new world.

Kubernetes provides a nice script to quickly deploy a small cluster on various infrastructure backends. On AWS, the documentation is a tad short about detailing how to customize the installation.

By default, Kubernetes is deployed on the Oregon availability zone (us-west-2a) with a cluster size of four minions driven by a single master. The EC2 instances for the minions and their master have a fairly limited size (t2.micro).

​Kubernetes relies on Ubuntu Vivid by default and favors Docker as the container provider. The container storage backend is AUFS which is the common option on Ubuntu.

Myself and David Dawson conducted tonight's inaugural Microservices Virtual Meetup panel discussion on how to fail, and succeed, at microservices.

The entire conversation was captured with audio and video and you can now watch/listen to the proceedings on YouTube.

The next virtual meetup is slated for October 1st and is on the topic of Deployment Infrastructure Options for Microservices. Spaces are limited so sign up now to avoid disappointment for this session.

In recent times I’ve been slightly shy of attending many conferences for a couple of reasons, the main one being that I found that I was learning less than I’d hoped.

Wondering whether I was just becoming jaded with things or not it was with some trepidation that I decided to attend the inaugural GOTO London conference next month to test whether it really was just me or whether it was something more in the industry.

Could I learn enough things that would be valuable to the journey I’m going on with the Antifragile Software book?

GOTO London felt like a good bet as, according to the “Why this conference” page there seemed to be an earnest desire to up the game for conferences.

I was also keen to see the new Skills Matter Code Node facility up and running in all its glory and since this is the location of GOTO London this was the perfect excuse for that too.

I then got a call to have a chat with Adrian Colyer, a member of the original programme committee responsible for making the conference what it is.

I’ve known Adrian for some years and he is one of the smartest and most interesting people in our industry, so I didn’t need much convincing that this chat was a good one to have.

Meeting Adrian in the beautiful offices of Accel Partners where he works currently as a Venture Partner after successfully helping to guide as CTO SpringSource to its acquisition by VMWare and then the transition to Pivotal, we had a great chat that happily traversed the topics of motorbikes (Adrian has two), the tech industry and its future and, of course, the forthcoming GOTO conference in London.

I had a blast chatting to Adrian and so here I’d like to share the facets that touched on the GOTO conference. 

Especially why this conference is an exciting experiment in upping-the-game for conferences generally!


----

Russ: "What’s most exciting about the conference for attendees? What are people hoping to grab from it in particular?"

Adrian: "The thing that is unique is that this is a highly curated conference.

Instead of a general call for papers and then just seeing what themes turn up, Adrian Cockroft from the start has been very involved and collectively we started with a singular question:

What’s interesting and happening in the world right now in software development and delivery?

Then we dug into:

What’s the story arc for that? What journey do people need to go on to understand things? Can we construct a conference that tells that story?

For this reason the talks are much more carefully selected and invited than usual.

We went for a single track rather than having the normal conference approach of, say, 8 tracks and you always feel like you’re missing out; wishing that you’d chosen the other talks; having almost too much choice!

With multiple tracks, nobody’s conference is the same conference and this severely hinders things.

We thought we’d go for a different and interesting approach: A single track, very curated story. The hope being that because everyone has been in the same session and that they move through them together you can have more of a conversation about that shared story than just the random hallway chats that usually occur.

If we’re all having the same journey together it should foster more interaction and discussion.

To me that’s an angle and a new emphasis and really caught my imagination.

If we can construct a good over-arching story arc and we can take everyone together as part of the same conversation that’s really interesting.

Russ: "So in many respects you’ve designed a conference to confer at?"

Adrian: "Exactly! If I want to see a potpourri of different talks from everyone else I have the videos, right? In that sense there is something special about this conference.

Jumbled threads hinder conversation, so we’ve disentangled the conference concept to enable better conversations and hopefully better journeys.

This to me is a really, really interesting experiment both in terms of taking an audience through this shared journey and the three themes involved: Lean, Agile and Rugged”.

Russ: “That’s a complaint I’ve heard, that this conference on the surface doesn’t look like it’s going to be really technical and, therefore, not valuable? Is that a fair criticism?"

Adrian: “I can see how people have got there from the terms, but in this case Lean and Agile really don't mean the methodology.

Instead the emphasis here is on what does it mean to have an Agile system? As in one that can respond to continual change? And adapt?

What does it mean to be the appropriate weight? Efficient resources? Be Lean?"

Russ: "The newer strand is the rugged one, and I admit when I too was making the mistake of assuming the titles of lean and agile were methodology-focussed Rugged confused me a bit! What’s Rugged all about?"

Adrian: "Rugged in our case means 2 things. Firstly it means systems that are resilient, in the face of unexpected, hostile change. Can it keep up and running, what are the best practices there?

In addition there is of course the security element too.

The three themes represent the attributes we would like a modern system to have and they inform the core flow of this curated, conversation-enabling, conference.

Russ: "That’s a nice point that may have been slightly missed when you glance at the conference page. It isn’t obvious. You see Agile and Lean and assume methodology. Then you see Rugged and you are surprised, but with the emphasis on desirable attributes that you’d want from a modern system, it all makes more sense and actively forces the conference into highly technical areas?"

Adrian: "Yes, Lean in this case doesn’t mean Kanban. This is more like ‘How do I build systems that are lean, agile and rugged. what do we have to do?'

Our challenge in designing the conference was to figure out how do we get these qualities, how do we get this story arc? We hope we’ve got that in GOTO London."

Russ: "What you’ve just described is a highly technical conference, and its being misinterpreted by the titles?"

Adrian: Yes, these are the higher-value, why-centric topics. While listing out the next release of the various technology buzz-words might seem more technical, these topics are essentially technical but tracked back to the ‘why we want them’ qualities in the first place.

Russ: "So you’re framing this conference journey with ‘why?’ This is something most conferences don’t do, there’s usually a lot of ‘how' and ‘what' instead. But once again, this isn’t really obvious from the conference overview?"

Adrian: "There are a few other things also added into the framework of the conference to support this.

There’s an introduction before every arc in the single conference journey that sets the stage, with different speakers then speaking on different perspectives on that quality, and then a retrospective panel session.

Now normally I’m not a big fan of panel sessions but in this case the aim is to bring back the speakers that have just spoken along with the person who has tee’d things up and now we can have a discussion with them and the audience that is really reflecting on the topic within the immediate context of ‘why’ we want those system qualities.

Russ: "It’s a great way to bring some additional reinforcement to the shared journey?"

Adrian: Precisely; to analyse it in different ways. Then hopefully to take that into the breaks and the lunch to enable those all important conversations.

Russ: "On the final day you seem to break from the single journey into three tracks. How does that fit into the curation of the conference?"

Adrian: "Yes at that point it does break into a more regular conference format. The talks touch on the themes and they are related but not necessarily part of the core conference story arc."

Russ: "So is it the case that during the first two days you’ll have built up, though the shared conference story arc, a shared understanding and comprehension of these qualities and the different perspectives on how they can be supported and achieved to then navigate this choice on day 3?"

Adrian: Yes, there’s now that shared context to, on the final day, begin to apply that thinking and context to evaluate current developments.

Russ: "If I was in a team and I’d seen the conference how would I convince others that I should go?"

Adrian: "Unlike a lot of conferences, this one has that answer there already: Hark back to the why’s, the three topic threads.

What is long-lasting value and will pay you back many times over will be thinking about these high-level issues, understanding the ‘whys’ having a way of evaluating what something is doing for you and whether it aligns with your goals or not."

Russ: "Giving you the tools to really think for yourself?"

Adrian: "Yes, precisely. A framework for thinking for yourself that will pay back many times over, beyond the details of release 1.7 of a particular project you might pick, for example."

If I’m the business I really don’t care about which particular projects or components have been used to build a system, but I do care about its amenability to change, is it resilient, is it cost-effective. Those high-level attributes that this conference has been built around.

Russ: "Sounds like the conference represents a compelling argument for any budget-holder?"

Adrian: "Precisely, that was the plan."

Russ: "Ok, so finally let’s talk about what you’re going to be talking about at the conference. Can you give us a sneak preview of what you have planned?"

Adrian: "I’ve got a really fun one actually! An evening keynote with the highly entertaining and smart Ines Sombra. She runs the "Papers we Love" SF chapter and has done some fantastic talks in the past.

Of course I regularly put out the Morning Paper where I go over the details of some academic IT paper that is perhaps extremely relevant to a number of current interests but has been somewhat swept under the carpet by the deluge of papers that are actually published in any given year.

So our keynote is going to be a Papers-centric talk that is about why papers could be more interesting and relevant than you could ever realise and how they should be a part of the toolbox of the thinking persons IT.

We’re going to pick a few selected papers inline with the conference’s themes that have really stunned us with their interesting and relevant research.

Ideally can we find one foundational, lean/agile/rugged, and then one frontiers paper to talk about where things are going to give the attendees to the conference a real glimpse of the possible futures.

We’re just debating what papers to pick and how to pull that off right now.

Honestly though I'm mainly looking forward to this talk because Ines is a ton of fun! Her talks are super-polished and deep. I’ve been doing keynotes for 10-15 years but Ines really ups the bar and so for me it’s fun to have myself pushed and my game upped too.

----

GOTO London is happening on the 16th to 18th of September, with workshops happening on the 14th and 15th September, and a few tickets are still remaining so get in quick and I hope to see you there and have some of those great conversations that Adrian was talking about.

I’ll be on the panel for this virtual meetup scheduled for Thursday 3rd September.

I’ll be sharing the multitude of experiences our clients have had in pitfalls to avoid with microservices, as well as how those pitfalls can be turned into success.

Being a virtual meetup this is the easiest to attend; I hope you can make it!

NOTE: Integration is a key challenge when working with Microservices and so is going to be covered in my book "Antifragile Software: Building Adaptable Software with Microservices" available today on LeanPub.





In this world of ever-smaller services and systems that need to evolve at vastly different rates, the challenge of integrating these services across various boundaries (organisational, political, technical, evolutionary) is rapidly becoming THE set of skills every software developer needs to master.


Unfortunately there is never just one answer to the question of integration; each context must be measured up against what is needed. This requires a lot of careful consideration, and so some guiding questions are needed to help lead you to your right answer.

I'm going to write a short sequence of blog posts on the challenge of integration to get you started on this journey, first by framing the whole issue with just 2 questions.

These 2 questions are the key concerns that should be constantly on the mind of every software developer when trying to integrate two systems, services or components. 

I'm going to use the 'system' terminology here for the two participants involved in the integration under examination because it is the most generically applicable term. Where I say 'system', you could replace it with service, application or component.

Those 2 crucial questions that should be on the mind of everyone considering integration are:

1. What do we need to agree between the two systems?
2. How much complexity are we will to buy to get the right level of flexibility on that agreement?

Question 1 is all about deciding what contract can be set between the two systems. Question 2 is about weakening that agreement to the right level.

For example it may be possible to accept RPC as an integration technology if the answer to question 1 is that we can agree to everything; we can be tightly coupled. We're going to share code, and evolve at exactly the same rates.

More likely is that one of those coupling factors needs to be weakened, and that's where Question 2 comes in. You could leap to completely asynchronous messaging to integrate your systems, but that comes with a whole plethora of complexity. My approach is to not  too much flexibility with complexity if you can avoid it, so I never start from the most flexible solution; I force myself to start from a brittle solution and weaken it.

Those two questions frame most of the details of integration, and in further posts I'll explore different scenarios where those questions will guide us to the right answers.