For a long time, I have been intrigued by the “rules” that guide someone in properly organizing the software code into classes and modules (aka packages, namespaces). I imagine that after applying those “rules”, one should obtain the same output (i.e. classes and packages/modules/namespaces) given the input (i.e. software requirements) is the same; it should be like in mathematics where given the input, e.g. a and b, the addition “rule” would lead to the same result, i.e. the sum of a and b. There are hints here and there, but I didn’t find anything convincing and complete in this regard so I decided to figure it out for myself; I describe my ideas in the following sections, and I hope you’ll find it useful.

In my investigation, I found the following books useful:

Design Patterns: Elements of Reusable Object-Oriented Software by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides is an inspiring and practical book
Domain-Driven Design: Tackling Complexity in the Heart of Software by Eric Evans is inspiring and important for my conclusion
Patterns of Enterprise Application Architecture by Martin Fowler has many useful sections even though some are outdated
Microservices Patterns by Chris Richardson is an inspiring and practical book

1. Problem

When writing software code, the main building blocks are the classes and modules (aka, packages or namespaces). Suppose the common case is for each class to be written in one file and a module is a directory containing such files. When working on complex software it might be difficult to determine what classes to create and how to group them into modules. A particular focus type is necessary to avoid the temptation posed by frameworks or technical aspects, to organize the software the wrong way.

PS: for a scripting language, e.g. JavaScript, the equivalent of a class could be a set of highly cohesive functions, placed in the same file

2. Layers and Modules

From the start, the most promising idea was to translate the layers, an application might have, into modules. Besides the books above, see also these excellent articles:

After reading at least the articles it should become obvious that it is not straight to get from layers to modules because the module’s structure looks like a tree while the layers are like a vertical set of rectangles or some concentric circles. Another problem is with the layers themselves: which one should be used?

3. The Border: External Interfaces and Adapters

One thing that is easy to spot is that an application (e.g., desktop, microservice, etc.) has a body (don’t mistake it for core) and a border; in this regard, I find Hexagonal architecture especially useful. The border is represented by all the application’s external interfaces plus the adapters through which it accesses the external services (exposed through external interfaces belonging to other applications).

Example of external interfaces:

RESTful endpoints
queue message listeners
WebSocket message listeners

Everything that asks for something from an application requires an external interface to talk to.

Example of adapters:

DAO (i.e. Data Access Object, e.g. the CRUD operations on the SQL or NoSQL databases)
Lucene index reader/writer (a kind of a NoSQL database)
stream reader/writer (e.g. Kafka topic reader/writer)
file system reader/writer
command line reader/writer
WebSocket message publisher or client
RESTful client

Everyone the application asks to do something requires an adapter.

The difference between the border and the body might become incredibly blurry when using a framework/technology. For example, Spring Boot can directly use the body based only on annotations, hence one could legitimately wonder where the external interface is; well, it’s still there though completely implemented by the framework. Change the application to provide an additional communication channel to the same body, one the framework doesn’t support, and the external interface will become visible.

3.1. From Border to Modules (examples)

a. Application exposing RESTful endpoints and persisting to a database

In this situation, I like to have in the application’s root these modules (directories):

datasource (SQL and NoSQL DAO classes)
rest (RESTful handlers and clients)
- if crowded, I might split its content into in and out modules

b. Application exposing RESTful endpoints, persisting to a database and using a messaging system (e.g. Rabbit MQ)

In this situation, I like to have in the application’s root these modules (directories):

datasource
queue (message listeners and producers)
- I might split its content into in and out modules
rest

c. A complex application providing a lot of external interfaces and using a lot of adapters:

In this situation, I like to have in the application’s root this structure:

datasource
- cache (in-memory database used for caching)
- dao
- index (Lucene index reader/writer)
- fs (file system, e.g. for loading/writing data from/to CSV or XML)
infrastructure (or io)
- mem (in-memory database used for something else than cache, e.g. distributed locking)
- queue
- rest
- shell (command line reader/writer)
- stream (Kafka topic consumer/publisher)
- websocket (message listeners and publishers)
- MVC controller

You can see that I prefer having the datasource module in the application’s root; this happens because, usually, there’s a lot of activity there. I guess the rule should be:

keep the most crowded external interface & adapter modules (EI&A) in the root while the rest into the infrastructure/io module. If there are too many EI&A modules in the root (including the infrastructure/io module) then use the infrastructure/io module to keep everything (for me, more than 3 means too many).

Additional module structuring hints:

use in (for external interfaces) and out (for adapters) modules inside infrastructure/io and/or its sub-modules
use consumer and producer inside queue (instead of in/out)
use listener and client inside rest, stream and websocket (instead of in/out)
use the event module for websocket, stream and queue; this works only if the event module is not too crowded
if too crowded, split a large module into smaller ones
name the modules closer to the technology, e.g. topic instead of stream for Kafka

d. UI Applications

The UI applications use fewer communication channels than a backend application, usually RESTful and WebSocket. Modules structure:

rest
- adapter for the external interface X
- adapter for the external interface Y
- …
websocket
- listener for the event E1
- listener for the event E2
- …
- message publisher for event P1
- message publisher for event P2
- …

If only RESTful/HTTP adapters are used, I might replace the rest module name with infrastructure or io. In both situations, if there are many adapters/listeners, I might group them into modules named by their source, e.g. the microservices they belong to.

3.2. Border Model

Data/messages are exchanged between the exterior and the application’s body; they could be very diverse and coupled to a particular infrastructure type; they are named DTOs (Data Transfer Objects) and constitute the body model. These DTOs might reach the body but in complex applications, they must be converted to the body structures. The corresponding DTO classes should be spread between the various modules presented before, but if used in common by two or more of them (e.g. queue and rest), then they could be placed into a new infrastructure sub-module, named model, domain or dto.

PS: don’t mistake the border model with the application’s model

3.3. Conclusion

Take a look at this crowded module structure:

Does it seem familiar (from a layout point of view)? I would say it looks like the contents page in a book; only by looking at it, can one understand the application’s capabilities (i.e. external interfaces) and dependencies (i.e. adapters). However, it might be misleading if one tries to learn the use cases it supports; the same use case could be provided through different external interfaces but over a distinct communication channel; a different “contents page” should be used for them, see it in the following section.

4. The Application’s Body

In previous sections, we split an application into border and body. The relation between them is like this: the border is like a set of doors through which messages pass between the clients and the body. The same message type could reach the body through a queue/topic (e.g. RMQ or Kafka) and a RESTfull endpoint; the body will have to react the same way, i.e. to perform the same work.

As one can observe the body is not a polyglot, it “talks” its language and the border must understand it! The border, on the other hand, is a polyglot, it “talks” many languages, e.g. JSON and queue’s “language”. This might not be obvious, especially when using frameworks that can automatically convert the outside-world messages to the body ones. This is fine while one doesn’t give up on the temptation of fitting the body “language” (this sounds funny 😃) into the border “languages”.

From a technical point of view, the messages exchanged between the layers, e.g. outside world and external interfaces or the external interfaces and the body, are all DTOs; they might have the same type/shape/class otherwise a conversion effort is necessary. Theoretically, there should be a large effort to “translate” the DTOs from one layer to another but in practice, the frameworks do it automatically hence the same DTO type can traverse multiple layers.

All those arrows pointing to the body form the use cases list, i.e. what the application can do. One might group many of them into one use case depending on what those arrows mean. The use cases list is very similar to a book’s contents page. For backend applications, I create a manager module where I put Manager classes dealing with those use cases. E.g. PlaylistManager is a manager class dealing with audio playlist management; the manager name is borrowed from Martin Fowler’s book.

4.1. The Manager module and classes

The manager classes orchestrate the activities performed by the adapters and the core; they won’t do business work but only decide who does what and delegate the work. The manager communicates with the client through the external interfaces and uses the adapters to accomplish its purpose (e.g. read/store something from/into DB). Besides the border and managers, the remaining part of the application is the core; usually, everything happening in the application’s RAM is a core activity (more about it later).

The messages between the managers and adapters are usually core objects but they could too be DTOs coming from the border or even from the outside world; it is the adapter’s job to understand them.

Be aware that DTO is a role; I name core object one used by the core but if passed between layers it’s a DTO too. There are also “pure” DTOs, e.g. a criteria object to query some DB never used by the core but directly passed by the manager to the DB adapter (e.g. a so-called Repository class). In this context, if the manager is doing nothing else but only to delegate to the adapter the temptation to skip it is huge. For PoC or small applications, one might give up on this temptation but if it does so, and some manager classes are missing from the manager module, later it’ll be hard to tell what the application does by only looking in the manager module (aka, the use cases list, the equivalent of a book’s contents page). A new developer might even duplicate some use cases because he isn’t aware of their implementation (aka, duplicated effort), another might change only one duplicated implementation (aka, partial fix), and so on …

The messages coming from the external interfaces might be used in common by many managers. In this situation I create for them a module named model or dto inside the manager module (it’s similar to the border model); it contains no type/class used by the core!

Examples of activities the manager might orchestrate:

load an audio playlist from DB, sort it (this part is delegated to the core), remove the duplicates (core), then store it back to the DB
periodically check town hall’s website for new building authorizations pdf documents, download them, extract their content then index it with Lucene (no core activity here)
listen for a queue message containing some payment transaction, load the actor profiles from DB, compute the fees (core activity), update the transaction details (core), store them in DB

4.2. The Core

The core is that part of the application deals only with the business it is supposed to solve but nothing else. If the business problem is about managing some audio playlist then the core would work only with concepts/notions/nouns regarding the audio playlists while excluding the rest, for example:

playlist (it has a name, a location and one playlist-entries object)
playlist-entries (it contains one or many playlist-entry objects)
playlist-entry (it has a title and a location, e.g. a file path or YouTube identifier, etc)

The core for the above example won’t deal with:

persistence
presentation
messaging systems
caching systems
locking mechanisms

and others. A good (but not perfect) technical hint is that the core will only use RAM to do its job.

One might notice that the core is small compared to the rest of the application; that’s true, and Eric Evans points this out too, in his DDD book.

If the application is small enough and a framework abstracting the infrastructure usage is used, the core might be overlooked altogether! For example, if the application is about extracting some data from the database (any type) and then sending it as JSON through a RESTful endpoint, nothing remains to do about the core when using a capable framework (pretty sad from my point of view).

On the other hand, if the application is complex the core would contain Entities, Value objects and Services (see DDD by Eric Evans). In this situation, on behalf of the core I create the model or domain module while inside it I usually create these modules:

one module for each Entity type
service (it deals with Entities and Value objects only!)
- on module per Service if Value objects associated with it exist (e.g. its operations’ parameters)

I put the Value objects inside the corresponding Entity or Service module otherwise directly inside the model/domain module; if too many, I group them in additional model/domain sub-modules and if those are many too, then I use a model/domain vo sub-module for them. Let’s visualize a crowded model/domain structure:

model (or domain)
- entity1
- …
- entityN
- vo
  - vo1
  - …
  - voN
- service
  - service1
  - …
  - serviceN

Though not forbidden, no relation is assumed between entity1, vo1 and service1.

TBC

adrhc's blog

How to organize the software in classes and modules (aka packages, namespaces)?