How to test the system using PostgreSQL and Testcontainers

To perform integration tests, we needed a real database server, which was not an optimal solution due to additional costs and inefficiencies. Using a different database for tests was not viable either. This led us to explore other solutions, and we discovered Testcontainers, an open-source framework that provides lightweight, throwaway instances of databases, which was precisely what we needed.

Our Technological Stack

• .Net
• PostgreSQL
• ABP Framework
• XUnit
• Docker

Our project utilises the ABP framework; you can check it here. However, our solution is generally framework-independent, with occasional use of ABP features.

Database creation

First, we need to create a database. Each test requires its own database to prevent interactions between tests. The PostgresDatabaseContainer class is responsible for database creation:

public static class PostgresDatabaseContainer { private const string DatabaseName = “test_db”; private const string UserName = “postgres”; private const string Password = “postgres”; private const string ImageName = “postgres:15”; private const int MaxConnectionsNumber = 20000; private const int ContainerAwaitDelay = 200;

private static Lazy<PostgreSqlTestcontainer> _container = new Lazy<PostgreSqlTestcontainer>(() => CreateContainer());
private static PostgreSqlTestcontainer Container => _container.Value;

/// <summary>
/// Creates new database and uses TDbContext object in order to create tables structure
/// Creates new docker container if needed
/// </summary>
public static IPostgresDatabase CreateDatabase<TDbContext>(Func<DbContextOptions<TDbContext>, TDbContext> creator) where TDbContext : DbContext
{
    var conectionString = GetRandomDatabaseConnectionString();
    Debug.WriteLine($"ConnStr:{conectionString}");
    var builder = new DbContextOptionsBuilder<TDbContext>();
    using var dbContext = creator(builder.UseNpgsql(conectionString).Options);

    dbContext.GetService<IRelationalDatabaseCreator>().Create();
    dbContext.GetService<IRelationalDatabaseCreator>().CreateTables();

    return new PostgresDatabase<TDbContext>(creator, conectionString);
}

/// <summary>
/// Updates existing database with TDbContext object in existing container
/// </summary>
public static void UpdateDatabase<TDbContext>(Func<DbContextOptions<TDbContext>, TDbContext> creator, string conectionString) where TDbContext : DbContext
{
    var builder = new DbContextOptionsBuilder<TDbContext>();
    using var dbContext = creator(builder.UseNpgsql(conectionString).Options);

    dbContext.GetService<IRelationalDatabaseCreator>().CreateTables();
}

private static PostgreSqlTestcontainer CreateContainer()
{
    if (_container.IsValueCreated)
    {
        throw new InvalidOperationException("Container was already created!");
    }

    var container = new TestcontainersBuilder<PostgreSqlTestcontainer>()
        // We don't need this database, it is used to expose port
        .WithDatabase(new PostgreSqlTestcontainerConfiguration
        {
            Database = DatabaseName,
            Username = UserName,
            Password = Password
        })
        .WithCommand("postgres", "-c", $"max_connections={MaxConnectionsNumber}")
        .WithImage(ImageName)
        .WithCleanUp(true)
        .Build();

    AsyncHelper.RunSync(() => container.StartAsync());

    // Give some extra time for container initialization
    AsyncHelper.RunSync(() => Task.Delay(ContainerAwaitDelay));

    return container;
}

private static string GetRandomDatabaseConnectionString()
{
    var randomDatabaseName = Guid.NewGuid().ToString();
    return $"Server={Container.Hostname};Port={Container.Port};Database={randomDatabaseName};User Id={Container.Username};Password={Container.Password};";
} }

As we delve into the CreateContainer private method, we can see how it creates a PostgreSqlTestcontainer using the TestcontainersBuilder. This class ensures that only one container is created, employing a lazy initialisation strategy. The first call to the ‘CreateDatabase method’ creates both a container and a database. Subsequent calls only create a new database. The ‘CreateDatabase’ method uses DbContext to create and update the database. The database name is unique, so that each test will have its own database. Note that this class returns an instance of IPostgres Database.

Database Lifetime

Each test benefits from having its own dedicated database, with the database lifetime spanning from the start to the end of the test. This is facilitated by the PostgresDatabase and DatabaseHelper classes, which we will delve into in the following section:

internal class PostgresDatabase : IPostgresDatabase where TDbContext : DbContext { private readonly Func<DbContextOptions, TDbContext> _creator;

public string ConnectionString { get; }

public PostgresDatabase(Func<DbContextOptions<TDbContext>, TDbContext> creator, string conntectionString)
{
    _creator = creator;
    ConnectionString = conntectionString;
}

public void Delete()
{
    var builder = new DbContextOptionsBuilder<TDbContext>();
    using var dbContext = _creator(builder.UseNpgsql(ConnectionString).Options);

    dbContext.GetService<IRelationalDatabaseCreator>().EnsureDeleted();
}

void IDisposable.Dispose()
{
    Delete();
} }

The PostgresDatabase class, previously mentioned, is created by the PostgresDatabaseContainer and contains the logic for database deletion. To effectively manage database lifetime, we utilise the ConfigureServices and OnApplicationShutdown methods from AbpModule, or any place in the code, that is executed before and after each test:

public class WithTestsBaseModule : AbpModule { private IPostgresDatabase? Database { get; set; }

public override void ConfigureServices(ServiceConfigurationContext context)
{
    ...
    Database = DatabaseHelper.CreateDatabase<MyDbContext>();	
}

public override void OnApplicationShutdown(ApplicationShutdownContext context)
{
    base.OnApplicationShutdown(context);
    Database?.Delete();
    Database = null;
} }

Configuring DbContext

All DbContext instances must be configured to use our newly created database. This is achieved through the DatabaseHelper class, which uses the PostgresDatabaseContainer class to instruct ABP to use the connection string of the created database:

public static class DatabaseHelper { public static IPostgresDatabase CreateDatabase(this IServiceCollection services) where TDbContext : AbpDbContext { var database = PostgresDatabaseContainer.CreateDatabase(options => Activator.CreateInstance(typeof(TDbContext), options) as TDbContext);

    services.Configure<AbpDbContextOptions>(options =>
    {
        options.Configure(abpDbContextConfigurationContext =>
        {
            abpDbContextConfigurationContext.DbContextOptions.UseNpgsql(database.ConnectionString);
        });

        options.Configure<TDbContext>(abpDbContextConfigurationContext =>
        {
            abpDbContextConfigurationContext.DbContextOptions.UseNpgsql(database.ConnectionString);
        });
    });

    return database;
} }

After the call to the CreateDatabase method, a new database is created within the Docker container, and all our DbContexts are configured to use it.

Summary

By integrating these classes with the ABP framework, we create a robust base for running integration tests. With everything properly configured and the database running in our Docker container, we can execute our tests seamlessly.

Utilising Testcontainers with PostgreSQL streamlines the setup of integration tests by providing isolated and consistent database environments. This approach not only reduces the overhead of traditional testing methods but also enhances the reliability and maintainability of our test suite. Incorporating Testcontainers into our testing strategy enables us to achieve more robust and efficient testing workflows, ultimately leading to higher-quality software.