I attended the SATURN software architecture conference in Baltimore the week of April 27. Microservices was an important topic at this conference this year. This is a summary of my notes from the various presentations on Microservices.
Microservices have been embraced by some high tech leaders (such as Amazon and Netfix), but microservices is not an automatic “silver bullet” to get better scalability, better performance, or faster deployment. Anyone planning to use microservices to develop new applications or scale up existing legacy systems will need to consider potential problems with deployment automation, versioning, service interactions, and automated testing.
There were four events at SATURN related to Microservices: a full-day workshop discussing research questions and practical questions about Microservices, a panel session on Microservices experiences which included a talk by Microservices pioneer Sam Newman, a “mock trial” of Microservices featuring Sam Newman and Simon Brown, and a presentation/discussion about the feasibility of using Microservices to support refactoring a legacy application.
Microservices is a design approach for building complex distributed systems. The system is divided into many small “services”, and the services are only allowed to communicate through inter-service messages. Unlike the SOA (Service Oriented Architecture) standard, service developers are free to choose any kind of message format or message channel. The result can a simpler, less bureaucratic, and less complex architecture.
Companies like Amazon and Netflix are using microservices widely, and there are many other companies that are looking to employ microservices to build new applications quickly or to port existing applications to a distributed cloud environment.
In the Microservices workshop, the discussion focused on five topics:
It is a distributed services architectural style, with the following constraints:
There are also two “organizational constraints” on the organizations contributing:
The goal of these constraints:
The most important conclusion from the workshop: Microservices are not for everyone or for every application.
Sam Newman presented a compelling case for using microservices in his panel session talk. Sam is the author of the new O’Reilly book Building Microservices
Sam gave an outline of seven top principles for building microservices:
You need to build clear boundaries around services and domains, because it makes it much easier to compose features. Sam recommends Vaughn Vernon’s book on Domain Driven Design.
A culture of automation makes it possible to move quickly. Sam’s case study was an Australian real estate company that used agile teams and automation to build their microservices – 60 microservices in 18 months. The main automation needs were in automating the services infrastructure and automating the testing.
The most difficult rule of microservices – you are not allowed to have two or more microservices that share data through a database. The database schema is an “implementation detail”, and there should be “ownership” of each section of the data model.
Deployment rules – it should be a rule that each item can be deployed by itself (with maybe some local coordination between small services).
One way to make things easier to test is to “codify expectations of each dependent service”. [Editorial comment: I think that this might be really hard. It requires more scenario-based thinking, not just API thinking.] Sam suggests using Pact as a tool for writing expectations. For more information about Pact, see https://github.com/realestate-com-au/pact
Isolating failure is a standard issue in any distributed system, and it is very important in microservices. We want to avoid having a “distributed single point of failure”.
The Microservices mock trial was an end-of-the-day event, with Sam Newman and Simon Brown taking extreme positions for and against microservices. There were a few good points in the hour:
There was a really useful discussion about microservices that was moderated by Einar Landre and Jorn Olmheim from Statoil in Norway. They are working on a 15-year-old oil well drilling management system – old, monolithic, ugly, and in desperate need of modernization. They are considering doing a microservices implementation that would refactor many of the existing applications.
The existing application is big: 3.5 million lines of code. It is mostly a PowerBuilder/Oracle application, originally built by a 3-5 person team, but the team has grown to 10 people. They envision a more clean separation of the architecture into user interface modules, separate APIs for various services, and separate domain models for each main area of the system.
They are working on applying Domain Driven Design (DDD) to model the responsibilities of each section of the system. They are certain that this is a much better way to redesign than to do a “data driven design” approach starting with the database tables and existing low-level data structures.
They also plan to use some of the standard data representation and transaction handling “patterns” from Martin Fowler’s Enterprise Architecture Patterns book. This should help make a flexible and extensible system.
Their assumption is that they won’t need a huge number of microservices – it is a lot smaller problem than the services Netflix is managing. They plan to use some simple “open protocols” for defining the microservice APIs – they are considering OData (Open Data Protocol – http://www.odata.org) or Atom (XML-based data format – http://en.wikipedia.org/wiki/Atom_%28standard%29).
There will be a lot of discussion of microservices in the future, especially as web-based and cloud-based applications are increasing. There will be a lot of chaos for a while, because it will take some time for tools, design methods, testing approaches, and deployment techniques to catch up. In some simple domains (e-commerce, web video portals), microservices are already making an impact.
Agile development approaches and microservices architectures will reinforce each other. If there is a need for “business agility” (deploying new variations fast), microservices will be seen as a way to get new product into the field quickly. On the other hand, the developers of high-reliability systems will probably avoid microservices for their most critical behavior – the development tools, testing methods, and deployment approaches are just not mature enough yet.
In some cases, there will be a technology push in the direction of microservices. We are starting to do more with cloud, containers, and Docker. One word of caution – think about whether an application really needs to use microservices… a microservices approach will require more development maturity, better automation tools for requirements, design, build, deployment, and testing, and more planning for how the architecture will evolve over time.