![]()
Plot Narrative – Agility
Nowadays, practically all organizations prioritize agility. Agility, or more particularly business agility, is a metric that measures how quickly an organization can respond to market changes, environmental changes, and innovative and emerging solutions. As a result, in the lack of business agility, a company will become insufficiently competitive.
Agility in IT refers to the ability to fulfill a new business demand efficiently and without regression. The only issue is the system's complexity intertwining. Complexity is an acquired characteristic. Thus, the purpose is to eradicate it.
Plot 1 – Monolith split, Microservices, Distributed Systems or Simply Bounded Context?
Monoliths are inherently prone to tight coupling. We might all have concluded that logically distinct components are preferable to a monolith. Please bear in mind that I am not discussing code level separations, and the notions I am introducing here are intended for use in systems or System of Systems. However, in a while, I will employ the terms "software" and "application" interchangeably to describe principles. In any case, reducing the conversation to loosen coupling of code components might lead to a variety of simple solutions for development purpose only!
If you too have already reached the conclusion described above, for splitting a monolith we should address the following questions (If you have not or believe otherwise, you still can continue reading as I am not actually debating which one is superior but only want to provide some ideas for ideas and to note that microservices architecture and monolith avoidance is extensively practiced in the industry today so we still shall address it.):
- How to properly establish the boundaries of a monolith's components?
- How should a monolith system's components communicate with one another?
Components created as a result of the monolith's split can contain any number of physical servers containing code (backend or frontend), jobs (such as schedulers), and data! These components can contain any number of databases with different schemas. Databases can be located on one or more physical servers and such.
Anti-Patterns
Let's examine the typical anti-patterns that occur in this domain.
Wrong Reuse
Noun-Services, or those that only perform CRUD tasks and hold no business logic, can be utilized incorrectly:
One of the key promises of distributed systems and service-oriented architectures is that we open up opportunities for reuse of functionality. With microservices, we allow for our functionality to be consumed in different ways for different purposes. Building Microservices: Designing Fine-Grained Systems
Hence, one should exercise caution when reusing a service and be concerned about how the system's static cast appears. It is also important to remember that service reuse can be counterproductive in some cases. Service reuse must be consistent with the Service's SOP (assuming one exists).
Also, ambiguous responsibilities must be prevented. Each service must have a distinct responsibility (or set responsibilities) that the consumer is aware of and completely understands. Otherwise, it jeopardizes the conceptual integrity of any system, which is its most important virtue. The main root of this anti-pattern is that Technology Architecture is inconsistent with the blueprints of Business Architecture in my view.
Synchronous Communications
Failure may stem from the following assumptions, commonly known as the "The 8 fallacies of distributed computing" (mostly when dealing with distributed systems):
- The network is reliable.
- The latency is zero.
- Bandwidth is unlimited.
- The network is secured.
- The topology does not change.
- There is one administrator.
- The transportation cost is nil.
- The network is homogeneous.
Services ought not to impede one another, either in point-to-point patterns or in Event sequences. More than one call in chain indicates a failure.
Asynchronous Communications
This appears to be a solution to technical issues related to synchronous communication, such as resiliency and resource costs. But simply technical ones; there are some more important issues, such as logical coupling which remains or being facilitated by async communications too.
Command Messaging Communications
Some may propose message systems and Command message as alternatives to HTTP calls due to their drawbacks. The resultant services are tightly connected, which is a common concern with command messaging and both synchronous and asynchronous communication. Again, let's keep things at the system level rather than the code level because I don't want to get into CQS and CQRS at the micro and macro levels, respectively.
Service and/or System-wide Orchestration
Service orchestration is the practice of managing several services and their dependencies in order to support loose coupling principles. Centralized governance should be avoided. In short, there is not enough room for message routing services.
Having Faith in Organizational Boundaries
An organizational structure is developed to address business issues. Quite frequently, people and politics have an impact on the Single Responsibility concept within that setup. Aside from that, business units may fluctuate, although an organizational structure is expected to remain constant or change slowly! Therefore, service boundaries that rely on this fragile and dynamic quality will be doomed to instability.
Services around Layers
If a system is separated into layers, the services are not connected. First and foremost, the volume of communication between service layers will remain constant, but it will be routed through the network instead. Second, from the perspective of organizational structure, that does not appear to be a wise decision; it will lead to the amount of cross-team communication and the likelihood of misunderstandings, which is considerable.
The Must Have Characteristics of Ideal Services
Yes, this is not the first place you might read this since it is well-known! But I am going to focus on some aspects further. Prior to that let’s have those “Google-able” attributes listed here:
- Single Responsibility
- Loose Coupling
- High Cohesion
- Autonomy
- Resilience
- Scalability
- Discoverability
- Interoperability
- Statelessness
- Security
- Observability
- Versioning
Now, let's look at the characteristics that services should have to be effective, scalable, and maintainable:
Low Coupling
Or Loose Coupling says that services ought to remain loosely coupled or connected, which means that changes in one service should have little – or none in ideal cases – influence on others. This enables each service to be designed, developed, deployed, and scaled independently.
If one service needs to be modified, it is likely that several others will need to be modified as well. Low coupling frequently results in strong cohesiveness, and vice versa. Surprisingly, the inverse is also true, but with a caveat given the appropriate service granularity, strong cohesiveness leads to loosen coupling.
It isn't a widely accepted phenomenon; it's simply an observation. It is also the one utilized to determine service boundaries, as services are loosely connected. This is a simplified depiction, as the concept of "loose coupling" appears too ephemeral to serve as a beacon for detecting service borders.
High Cohesion
Allow me to define "cohesion" so everyone understands. Here is what Wikipedia has to say:
…cohesion measures the strength of relationship between pieces of functionality within a given module.
The functionality of a service should be closely related to ensure that it is self-contained and cohesive. A service with a high level of cohesiveness is easier to comprehend and manage.
Conversing about service coherence demands the same approach — “coherent piece of functionality”. And this is what Cohesion is striving for.
Note: The English word "cohesiveness" refers to the degree to which a piece of text is unified with respect to its constituent parts ("and", "therefore", "moreover", etc.). "Coherence" refers to the degree to which the text is logically sound. However, they are frequently used synonymously in this technical context.
Right Granularity
One of the techniques which I am focusing on – too like many others possibly - is based on the concept of Bounded Context. In my experience it was one of the most efficient techniques for this purpose. If you are studying a service for its granularity, you can start with checking whether:
- some concepts are growing ambiguous (who is customer? The one who browses the page or the one who has made a purchase?),
- the same concept is used in more than one context semantically which interests in different “data” and “behavior” (for instance “order” while purchasing and “order” while delivering are identical semantically, but we know they demand different behavior and data),
- the same entity reflexes different domain concept in various stage of its life cycle (both examples above work here)
So, if the check asserted positive the service you are assessing is probably too coarse-grained and lacks cohesiveness. It is strongly advised to separate it further to create more cohesive segments. Once the result is services with mono-semantic limited contexts in which all ideas are unambiguous, no additional splitting is required.
High Autonomy
Services should function independently of one another. Each service should manage its own data and maintain its own database to reduce dependencies and shared states.
Service autonomy refers to a service's capability to carry out its work without relying on the availability, functionality, or data of other services.
Autonomy should be exercised purposefully by establishing appropriate service boundaries; strong autonomy leads to high business agility.
Service autonomy is manifested in communication through events and decentralized data storage.
Events not Messages
Services cannot exist in isolation; they must communicate with one another. The use of behavior-centric and business-driven event message types (known as Event-driven architecture), as opposed to synchronous requests and command messages, is a good implementation.
Published events must correspond to business concepts and actual events in the domain, such as a completed sale, a transaction performed, or an invoice paid. If the event should be processed within the database transaction, with the result being "all or nothing, and immediately" — reconsider the boundaries and consequently the service’s responsibilities!
In some circumstances, a service is fundamentally request/response-like. The aim here is for scenarios in which this is truly a separate service, reflecting some business value at the same abstraction level as the others, and so cannot be integrated into any of the current services. In this instance, command messages or asynchronous request/reply are unavoidable, but only in a non-blocking manner or as a fire-and-forget.
Decentralized Data
We established that Service autonomy implies that a shared database is not possible nor desirable.
If the service responsibilities are well specified, and each service is highly coherent, they should not require data from other services. However, if one service requires the data of another, which is a synchronous activity by definition, they should most likely be merged.
It is comparable to Feature Envy Smell, which is a clear breach of the GRASP rules. It's simply another illustration of the concepts of unity on multiple fronts and choosing between an object or a service.
Service Chorography
Imagine a ballet. Each one of the dancers knows everything they need to accomplish and how to engage with others. All occurs instinctively, with absolutely no need for an orchestrator. As mentioned earlier, to support loose coupling principles, centralized governance should be avoided.
In our context, choreography implies the orchestration of interactions among various components or systems where the components or systems interact with each other through a set of well-defined interfaces, without the need for a central authority or orchestrator.
Service choreography is a logical result of discarding synchronous communication, utilizing business events, and storing data decentrally.
Practical decoupling by employing Bounded Contexts
So far, I hope it is clear where I am headed. There is an urgent need for a shift throughout thinking, and I am attempting to provide some clues in this piece of writing. It seems that some still confuse decoupling and separation of concerns with deployment model. Prefixing the services with Micro will not help. It is crucial to consider bounded contexts rather than service sizes. Business should come first, not architecture! In TOGAF terminology get the Business Architecture right before Technology and Application Architecture. I will dive into Bounded Context and DDD more in detail in upcoming plots.
Plot 2 – Data Architecture and Application Architecture Define One Another
In this plot, I hope to refresh the TOGAF framework and clarify its position in relation to the symbiosis of Application and Data Architecture. The Open Group Architecture Framework (TOGAF) outlines extensive principles for establishing, conducting, and sustaining enterprise architectures. TOGAF explicitly defines the association between Data Architecture and Application Architecture, which is critical for enterprise architecture alignment and integration (as I mentioned earlier in this write up, the viewpoint is enterprise not a single application of software architecture). Here's how TOGAF advocates addressing this relationship in brief:
Separation of Concerns
- Data Architecture concerns the structure of an organization's logical and physical data assets, as well as its data management resources.
- Application Architecture explains individual applications, their interconnections, and their ties to the organization's fundamental business processes.
TOGAF Standard, Version 9.2, Part II: ADM, Phase C (Data Architecture) and Phase C (Application Architecture).
Interdependency and Integration
- Data Architecture establishes the fundamental data structures that Application Architecture use. Applications consume, manipulate, and generate data. This is known as Data as a Foundation.
- The Application Architecture must ensure that data is accessed efficiently and securely. Applications must follow the data standards and policies outlined in the Data Architecture (Data Access Management)
TOGAF Standard, Version 9.2, Part IV: Architecture Content Framework, Data Architecture and Application Architecture chapters.
Alignment and Consistency
- Application data model consistency tends to guarantee enterprise-wide data consistency and integrity, applications should conform to the data architecture's common data models, entities, and relationships.
- The data services should be put in place so that applications may utilize them to manage and access data consistently. This includes data warehouses and master data management.
TOGAF Standard, Version 9.2, Part II: ADM Guidelines and Techniques, and Part IV: Architecture Content Framework.
Interoperability and Reusability
- To make sure that various apps can work together and communicate data without any problems, we need to establish and adhere to common data formats, protocols, and standards.
- Encourage the development of data components and services that can be reused across many applications to decrease data redundancy and guarantee data quality also known as reusable components.
TOGAF Standard, Version 9.2, Part IV: Architecture Content Framework, and TOGAF Library.
Governance and Compliance
- Data Governance guarantees data privacy, security, and quality by applying data governance standards uniformly across all applications.
- Data governance policies must be followed by applications to make sure that data management satisfies organizational and regulatory requirements.
TOGAF Standard, Version 9.2, Part II: ADM, Phase G (Implementation Governance).
Change Management
- Impact analysis should be used to evaluate how changes to data structures affect applications and vice versa. Maintaining alignment and synchronizing modifications is necessary to ensure minimal disruption.
- Version control techniques should be applied to both data models and programs to manage changes effectively and maintain backward compatibility.
TOGAF Standard, Version 9.2, Part II: ADM, Phase E (Opportunities and Solutions), and ADM Guidelines and Techniques.
Lifecycle Management
- Handling the Data Lifecycle (production, archiving, usage, storage, and removal of data) shall be in accordance with the application lifecycle.
- In order to maintain their effectiveness and relevance, applications must be developed and managed with the underlying data lifecycle in mind.
TOGAF Standard, Version 9.2, Part II: ADM, Phases C (Data and Application Architectures) and Part IV: Architecture Content Framework.
Tools and Repositories
- Integrated architecture repositories can be used to store and manage application artifacts as well as data, providing a single source of truth and promoting improved alignment and collaboration.
- To help with planning, analysis, and visualization, use modeling tools that can depict application and data architectures as well as their interdependencies.
TOGAF Standard, Version 9.2, Part IV: Architecture Content Framework, and TOGAF Library.
Essentially, what I've tended to show here is TOGAF ADM Phases C, D, E, and G. By adhering to these recommendations, TOGAF guarantees that Data and Application Architectures emerge in a coordinated and integrated manner, offering a strong basis for enterprise architecture.
Plot 3 –Domian Driven Design (DDD) In Vogue or Vintage?
As usual let’s start with a loaned definition:
Domain-Driven Design (DDD) is a software development paradigm that emphasizes the need to comprehend and represent the business domain. This is an approach for increasing software quality by better aligning it with the business requirements it serves. DDD been introduced in 2003 by Eric Evans’ book "Domain-Driven Design: Tackling Complexity in the Heart of Software." The book (popularly known as the blue book; or maybe you are familiar with the Red one), provides an inventory of patterns and practices that have been nurtured and broadened by a CoPs throughout the time:
Returning to the definition! At its heart, DDD focuses on overcoming complexity by focusing software development on the 'domain', or the specific business context in which the final product works. It emphasizes the adoption of a 'ubiquitous language', which is a common language shared by both software engineers and business stakeholders. This language, employed for software design and implementation, assures that the software matches the business domain for which it is intended. Overall, I believe that DDD has become widely adopted in architecture practices.
I'm not going to go into detail about DDD here because the goal isn't to add another DDD document to the thousands (if not more), but rather to serve as a reminder and to make the plot rich enough to bounce the next plot into.
So, let’s quickly investigate the concepts, principles and building blocks. Where necessary I will expand it withing regarding sections.
I - Concepts and Principles
Core Domain
Each business has a core area that drives operations and determines success. DDD emphasizes identifying and focusing on this crucial domain. Architects learn business’s fundamentals by delving into the core domain's intricacies. This understanding allows for applications that meet the business's specific needs.
Model Driven Design
DDD recommends a well-defined domain model to connect the business domain and software. This model conceptualizes the business domain's basic parts, entities, and relationships. This paradigm - created with domain experts - guides application development. Architects and software engineers use the domain model to guarantee that their application matches the business it supports.
Ubiquitous Language
Effective communication is key to software development success. DDD uses a ‘ubiquitous language’ to connect architects, developers, domain experts, and users. From conversations and documentation to implementation, this shared language is used throughout the software development process. This guarantees that everyone understands the business domain, making collaboration easier and lowering the chance of misconceptions.
Let’s expand a bit further.
This is where Language has a highly specific meaning inside a Context. Before anything else, having a common language (Ubiquitous Language) with all parties involved (businesspeople, architects, engineers etc.) is critical. Here's a traditional example of a well-known object; “Shirt”. Let’s ask few questions about this object:
- What is the Shirt definition?
- What is Shirt?
- What is Shirt used for?
Before you proceed further with this note, try to answer all in your mind.
Everyone should have a definition and answers to those questions based on the occasion, cultural background, taste etc.
Image credited in References
Obviously here the assumption was that Shirt is used by “Human” and the sample image shows the variety of shirts again with another assumption that by “Human” we meant “Women”. Our language was not precise in the context! Let’s change the context of the questions from what’s been assumed “Shirts for Human” to “Animals”! then we have totally a new story:
Image credited in References
Once the context switched more assumptions can be validated. For instance, will it cover the animal’s butt too?
Image credited in References
And one can continue to play this game! This is a pretty common scenario among software engineers, developers, architects, analysts, and even product owners. They deal with many contexts and languages in an ever-changing business environment with businesspeople from diverse cultures, knowledge, and experience, as I previously stated. To that end, "Ubiquitous Language" is crucial to get everyone involved in same page about a specific concept (in above instance, “Shirt” should have a unique and agreed definition among parties).
Iterative Collaboration
Application and more generally Software development is not a one-time deal. Software must change with business. Iterative collaboration between technical and domain expertise is encouraged in DDD. The domain model and software are improved by this constant feedback. Developers may keep their software relevant and productive by maintaining connections to the changing business sector.
II - Building Blocks
Bounded Context
In sophisticated systems, the business domain may have multiple components with different rules and demands. A domain model applies to bounded contexts, which create logical limits in the system. Each bounded context has its own ubiquitous language, which gives system parts autonomy and adaptability. This breakdown lets teams concentrate on their respective domains without distractions from unrelated areas.
I’m going to expand this Essential Building Block (EBB) further.
Domain-Driven Design's essential building block is Bounded Context. It is the core of DDD's strategic design for massive models and teams. DDD splits massive models into various Bounded Contexts and specifies their relationships. DDD is all about designing based on models of the underlying domains.
Speaking of Devil! Models! Model identification is crucial to Bounded Context and, generally, implementation of DDD. The objective is to maximize frequent delivery and put the feedback loop in place (isn’t this CI/CD?). Model definition requires iterative actions and should be examined:
- Define different models
- Justify why they are right models
- Redefine models
Iteration is just necessary, as George Box clearly explains:
All models are wrong, but some are useful.
Model selection made much easier with the use of the Theory of Constraints principles:
- Convergence
implies that a complex system will be easier to manage because any changes made to one component will have an effect on the entire system for the better.
- Respect
indicate that despite their flaws, people are generally decent and worthy of respect even when they make mistakes. Because error occurs when humans are involved, this gives project management the much-needed discretion and flexibility.
A few brief tips for Bounded Context discovery but expanding them is beyond the scope of this note:
- Business Capabilities
- Business Processes
- Business Services
- Linguistic Boundaries
- Data
- Uniqueness
- Flow
- Dependency
- Ownership
- Consensus
- Domain Experts
- Existing Boundaries
- Adopting “-tion” and “-ing”
- Clusters of Function plus Data
Practices like Event Storming will be useful.
“Boundaries are always subject to negotiation”
Alanna to Hannibal
Hence, keep eyes open and check for symptoms of boundary changes:
- Bottlenecks and Blockers
- Development time
- New business aspect
- New domain
- Technical insights
- Show and tell
- Cross team pairing
- Cross functional pairing
- Event storming
- Code Commits
- Traffic between Services
- Near Synonyms
I'd like to underline the characteristics of microservices that were described earlier because they are relevant when we talk about bounded contexts. Having complete autonomy, low coupling, and so on are all aspects of a strategic design decision, and they have repercussions. Bounded Context decides to reduce coupling at the expense of (Data) redundancy! However, in the course of time, they will evolve differently.
And to be clear the answer to all below questions is NO!
- Is it mandatory to keep a 1 to 1 relation between Bounded Contexts and Microservices?
- A Bounded Context must not include user interfaces/interactions?
- Bounded Context is a low-level coding pattern?
Entities
Certain objects in the outside world have a particular identity that lasts throughout modifications. Entities represent software objects in DDD. Entities' identity defines them, unlike properties. For instance, in a customer management system, a customer has a persistent identity independent of name, address, or other changes.
Value Objects
However, some domain entities are defined by their properties rather than their identity. DDD calls them value objects. Immutable value objects are unable to change after constructed. They may be replaced with another instance with the same properties without affecting the system. Date ranges, addresses, and money are valuable objects.
Aggregates
Entities and value objects often depend on each other in complex scenarios. Aggregates cluster entities and value objects into a single unit. All interactions with an aggregate occur through its aggregate root. Aggregates ensure that cluster modifications are coherent and follow domain rules and constants by maintaining the consistency and integrity of connected items.
Domain Events
Certain business events are significant and may cause system-wide reactions. Domain events illustrate these crucial moments from the past. These events are broadcast so the software may react and respond, sustaining system integrity and coherence as the business domain evolves.
Plot 4 – The Denouement or The Alignment of DDD, Microservices and TOGAF Guidelines
DDD, Microservice Architecture, and TOGAF are three distinctive but supportive approaches to application and data within enterprise architecture scheme. Below are some observations which involves all three and their similarities in approach and/or recommendations:
Observation #1: Establishing Boundaries
In DDD Bounded Contexts define clear boundaries within which a domain model is valid. Following that, each Microservice can be attributed to one or more Bounded Contexts, encapsulating a specific or set of business capabilities. This is in line with the concept of establishing clear boundaries for applications and data in TOGAF , which ensures that each application component serves a particular business purpose.
Observation #2: Data, Ownership and Autonomy
Every Bounded Context has its own model and data management, eliminating reliance on other contexts, and each service (in Microservice environment) also owns its data store, ensuring autonomy and lowering the risk of data inconsistencies. This is exactly what TOGAF highlights; the significance of clearly identifying data ownership and management roles in the Data Architecture.
Observation #3: Data Entities, Entities and Value Objects
DDD distinguishes between entities (with unique identities) and value objects (specified by attributes); in Microservices, services should be designed so that each service manages specific entities and value objects, ensuring a clear separation of concerns. TOGAF supports this by recommending that data entities and their relationships be defined within the Data Architecture to correspond with business processes.
Observation #4: Entities Aggregation
In DDD, aggregates cluster related entities together, and repositories allow access to aggregates, as each Microservice can handle one or more aggregates, resulting in a focused and cohesive business process. The TOGAF guideline, in support of the same goal, assures that application components (such as services) are designed to handle specific data entities and processes while remaining consistent with the overarching data strategy.
Observation #5: Business Objectives Alignment
DDD points out an in-depth comprehension of the business domain and aligning the software (or application) model with business goals, whereas Microservices concentrates on delivering business capabilities via independent, loosely coupled services, and TOGAF promises that both data and application architectures are in line with the organization's business strategy and goals.
Let's Summarize
In summary, there is correlation between Domain-Driven Design concept, Microservice Architecture, and TOGAF guidelines. They all emphasize the importance of establishing clear boundaries, data ownership and autonomy, and the use of entities. They also prioritize aligning the software model with business goals and ensuring that both data and application architectures support the organization's business strategy. Overall, these concepts and principles work together to create a cohesive and effective enterprise architecture.
..and for reference
https://tidylab.github.io/microservices/
https://raw.githubusercontent.com/tidylab/microservices/master/README.Rmd
https://www.logmemo.dev/blog/2021-04-01-building-microservicies-by-sam-newman
https://www.linkedin.com/pulse/how-tech-startups-might-benefit-from-microservices-galym-akishev-elsyc
https://www.rdocumentation.org/packages/microservices/versions/0.2.0
https://stackoverflow.com/questions/34903605/microservices-what-are-pros-and-cons
https://www.oreilly.com/library/view/building-microservices/9781491950340/ch01.html
https://hackernoon.com/wrong-ways-of-defining-service-boundaries-d9e313007bcc
https://medium.com/@wrong.about/wrong-ways-of-defining-service-boundaries-d9e313007bcc
https://github.com/mahmudaAfreen/Creating-an-e-Attendance-System-with-Python
https://hackernoon.com/what-characteristics-my-services-should-possess-ca22294bbea6
https://medium.com/@wrong.about/what-characteristics-my-services-should-possess-ca22294bbea6
https://www.bitcoininsider.org/article/14381/what-characteristics-my-services-should-possess
https://redis.io/glossary/domain-driven-design-ddd/
Also images Credits
https://fashionista.com/2019/12/ashley-williams-best-friends-dog-t-shirt
https://www.pinterest.ca/pin/ultimate-guide-to-sewing-clothes-types-of-shirts--91479436172331555/
https://petsmedic.au/products/mps-medical-pet-shirt%C2%AE-cat
Cheers,
Mohammad Malekmakan
Disclaimer:
All opinions and content published in my blog and my social networks are solely my own, not those of my employer(s) and the communities I am contributing in.