Abstracting dependencies to keep a flat dependency tree between components

Every piece of software has dependencies. No one likes to reinvent the wheel.
So when you need a database access library or logging, using an existing one is a given.

When these libraries have dependencies of their own, you become dependent on them as well.
To give a concrete example: Nest, an elasticsearch client for .Net, depends on Newtonsoft JSON which is a JSON manipulation library. So if you are using a different version (newer or older) of Newtonsoft JSON, you now have a problem.
This simple example includes two levels of dependencies. Use any module in NodeJS from NPM and you'll get at least three level of dependencies – same for every platform you can think about.

Today, I want to discuss a nice trick we are using in order to keep the dependency tree flat, making libraries more maintainable and easier to use.

Example

Lets say you are building a data access library. One of the features you like to add is performance logging – in case query runs too long, log it (for monitoring, production time debugging, you name it – its quite useful).

The naive approach is to simply take dependency on some logging framework (NLog for example), and use it when you want to log the query:

public class Storage<T>
{
   private readonly Logger _logger = LogManager.GetCurrentClassLogger();
   ...
   public IList<T> Query(...)
   {
      var stopwatch = Stopwatch.StartNew();
      try
      {
         ...
         // Execute query
         ...
      }
      finally
      {
         var elapsed = stopwatch.Elapsed;
         if (elapsed > 500)
            _logger.Warn($"[Perf]: Query ran {elapsed} ...";
      }
   }
   ...
}

A more common approach is to use one of the logging abstractions out there (line Windsor logger), and use that instead. Now developers will not depend on NLog, but they'll depend on Windsor:

public class Storage<T>
{
   private readonly ILogger _logger;
   
   public Storage<T>(ILogger logger)
   {
      _logger = logger;
   }
   ...
   public IList<T> Query(...)
   {
      var stopwatch = Stopwatch.StartNew();
      try
      {
         ...
         // Execute query here
         ...
      }
      finally
      {
         var elapsed = stopwatch.ElapsedMilliseconds;
         if (elapsed > 500)
            _logger.Warn($"[Perf]: Query ran {elapsed} ...";
      }
   }
   ...
}

Both options present the same problem: Create a second level dependency on a library the user might not want to use, or use a different version of it.

Another interesting problem here is that our cool feature is somewhat hidden from the developer – it will surprise him when he starts to use our library.
What if the developer doesn't want to log it under warning, or doesn't want logging at all? We can do these with some kind of configuration system, but as the scenarios become more complex so does our configuration.

DEPENDENCY HELL

Occurs when a library/module/app becomes dependent on second+ level dependencies from libraries it uses, making it hard to update or use different versions of them.

When you are the one building these libraries, it can become an even bigger problem – what will happen if you need to change a dependency on the top of the tree? How many updates to the underlining libraries will you need to do?

Usually, you don't think about these kind of things if you have only one library you maintain. When we started managing most of our code as libraries (a post on that to follow), we stumbled upon these issues very early in the process.

Quickly, our guideline became:

Avoid deep dependency trees between different components, preventing dependency hell.

Application level dependencies

Of course you'll need to use external libraries – it cannot be prevented and nor should it. What we can do is to move these dependencies to the application level instead of the library level.

When building a library, instead of using dependencies directly, expose a contract or interface needed to perform the action, and make the application using this library provide the implementation for this contract.

There's a name for this pattern – Inversion of Control or IoC.

Instead of doing IoC on a class level, which we usually do, we now do it on a module/library level.

Example

Instead of using another library, let's define what we actually want to do:

public interface IPerformaceLogger
{
   void LogPerf(string query, int queryTimeInMilliseconds);
}

Now let's use it:

public class Storage<T>
{
   private readonly IPerformaceLogger _perfLogger;
   
   public Storage<T>(IPerformaceLogger perfLogger)
   {
      _perfLogger = perfLogger;
   }
   ...
   public IList<T> Query(...)
   {
      var stopwatch = Stopwatch.StartNew();
      try
      {
         // Execute query here
      }
      finally
      {
         var elapsed = stopwatch.ElapsedMilliseconds;
         if (elapsed > 500)
            _perfLogger.LogPerf(query, elapsed);
      }
   }
   ...
}

And now the developer using the library simply needs to implement IPerformaceLogger and log it however he wants! Let's take a look on the application side:

public class MyPerfLogger : IPerformaceLogger
{
   public void LogPerf(string query, int queryTimeInMilliseconds)
   {
      Console.WriteLine($"[PERF]:\t Query running ${queryTimeInMilliseconds}:\t ${query}");
   }
}

public class Program
{
   public static void Main()
   {
      var storage =  new Storage<MyDto>(new MyPerfLogger());
   }
}

A nice artifact we get is awareness of the feature while coding, making it more discoverable. A great added bonus!

Usability

Abstracting dependencies like we did has many advantages, but it also has some problems, mainly from a usability point of view.

Boiler-plating

It makes it harder to use the library since it requires more boiler-plating. Consider a high level library that has 10 such contracts you need to implement. Its quite annoying.
On the other side of the coin, you can imagine the DRY issue – several users will implement the same abstraction the same way.

We use a couple of tricks in order to deal with this

Optional implementations

Logging in a library is an excellent case of optional dependency. A developer using the module might or might not want to use your logger. In this case, providing empty/default implementations to the contract will allow the developer to opt-in whenever he wants.

public interface DefaultPerformaceLogger : IPerformaceLogger
{
   public void LogPerf(string query, int queryTimeInMilliseconds)
   {
      // Do nothing
   }
}

...

public class Storage<T>
{
   private readonly IPerformaceLogger _perfLogger;
   
   public Storage<T>(IPerformaceLogger perfLogger = null)
   {
      _perfLogger = perfLogger ?? new DefaultPerformaceLogger();
   }
   ...
   public IList<T> Query(...)
   {
      var stopwatch = Stopwatch.StartNew();
      try
      {
         // Execute query here
      }
      finally
      {
         var elapsed = stopwatch.ElapsedMilliseconds;
         if (elapsed > 500)
            _perfLogger.LogPerf(query, elapsed);
      }
   }
   ...
}

Of course you can wire these implementations with an IoC container instead of these null checks, but you get the point.

Bridging libraries

To solve the DRY issue, and also make it easier to use the library, consider creating bridging libraries that implement common usages developers might use. It is a great way to ease into using the library. Or wait for the internet to do it for you :)

On the data access example, it might be nice to create a ConsolePerformanceLogger or EventLogPerformanceLogger for the simple use cases, and additional libraries that can do the same with NLog, log4net and windsor. Those will contain the configuration needed from them (like log level).

You might say that we are now stumbling with the same dependency issue as before – now the app is now dependent of NLog, log4net and so on.

The difference is you don't have to use them in order to use the core library. They are just helpers giving the developer a choice. And if none of them fit, due to versioning issues or others, don't use it – implement it on your own!

Move the choice from the library developer to the application developer

IoC out of the box

While IoC is common on most production applications, when you want to start something quick, if it's either a POC or just a playground to test something, you usually won't start with an IoC container.
Here we kinda enforce IoC on applications using the library (which is not a bad thing), but it will make it harder for developers in the POC scenario to use.

There is no magic bullet for this one, you just need to accept it as a drawback.

Our workaround was to create a library that will make it super-easy to integrate an IoC container to an application, facilitating usage of our components even in these scenarios.

Epilogue

Abstracting dependencies in libraries, creates a maintainable eco-system of components, allowing them to be composed by the application with minimal friction to developers.

By using this simple method, we are making our libraries easier to update, and remove coupling to other, probably unrelated pieces of code. We're creating a library that is easier to integrate within existing apps, not binding the developers to other libraries they may not want or need.

Abstraction works pretty well in most cases, but there are more complex scenarios that need a bit more work. I'll discuss them at a different time.

Happy coding!