Managing Baseline creation in Applications based on ABP.IO framework

Why do we need „Baselines”

In one of our recent projects, we were tasked with designing a module to create baselines that provide clear, stable snapshots of application data at specific points in time. This is essential for historical reference and analysis. Data versioning allows for detailed tracking of changes over time, enabling users to see what modifications have been made and when.

Architecture and Implementation

Baseline Class

A key component of the “Baselines” is the „Baseline” class.

public class Baseline : FullAuditedAggregateRoot<Guid>, IMultiTenant
{
    public string Name { get; protected set; }
    public string Description { get; protected set; }
    public Guid? TenantId { get; protected set; }
    public Guid DataVersionId { get; protected set; }
    public int Number { get; protected set; }
}

The „Baseline” class encapsulates a reference point in an application, crucial for version control and change tracking. The crucial property „DataVersionId” stores a unique identifier of the data snapshot which will be saved on all versioned business objects in the application

IVersionedEntity Interface

A key component of the versioning module is the „IVersionedEntity” interface. It is designed to manage the versioning of business entities in an application, providing a structured way to handle different snapshots of data objects.

public interface IVersionedEntity
{
    EntityVersion Version { get; }
    Guid VersionId { get; }
    bool IsCurrentVersionedEntity { get; set; }
    Guid VersionedEntityPrincipalId { get; set; }
    void SetEntityVersion(Guid id, bool isCurrentVersionedEntity);
    void SetEntityVersion(Guid id, bool isCurrentVersionedEntity, List<Guid> versionedEntityVersionIds);
}

The „EntityVersion” class is a crucial component for managing business entity versions in an application. It tracks whether the entity version is current and maintains a list of related data version IDs. (baseline DataVersionId)

public class EntityVersion :
    AggregateRoot<Guid>,
    IHasCreationTime
{
    public bool IsCurrentVersionedEntity { get; set; }
    public List<Guid> VersionedEntityVersionIds { get; set; }
    public DateTime CreationTime { get; set; }
    public Guid? VersionedEntityId { get; set; } //technical field for generation initial values when versioning some entity
}

How it works?

Each time a new business entity record that implements the “IVersionedEntity” interface is created, an additional “EntityVersion” entity is also created in relation to the versioned business entity. This “EntityVersion” entity stores information indicating that it is the current version (IsCurrentVersionedEntity = true) and does not have a saved snapshot identifier yet.

When a user wants to create a snapshot of business data in the system, a “Baseline” entity is created using the application service, and a snapshot identifier (DataVersionId) is automatically generated for it. Using the “LocalEventBus” a native SQL update is triggered for all current (“IsCurrentVersionedEntity” = true) „EntityVersion” entities (and consequently the related 1-to-1 business entities) with the snapshot identifier.

public async Task SetVersionOnVersionedRecords(Guid dataVersionId)
{
    var dbContext = await GetDbContextAsync();
    var entityType = dbContext.Model.FindEntityType(typeof(EntityVersion));
    if (entityType != null)
    {
        var schema = entityType.GetSchema();
       var tableName = entityType.GetTableName();
        var sql = $"UPDATE  \"{schema}\".\"{tableName}\" SET \"{nameof(EntityVersion.VersionedEntityVersionIds)}\" = \"{nameof(EntityVersion.VersionedEntityVersionIds)}\" || ARRAY['{ dataVersionId }'::uuid] WHERE \"{nameof(EntityVersion.IsCurrentVersionedEntity)}\" is true";
        await dbContext.Database.ExecuteSqlRawAsync(sql);   
   }
}

Saving Changes

The version of a business entity which in fact is a copy of the original entity is created only at the moment of editing or deleting an already baselined entity using „SaveChangesAsync” on the custom „DbContext” class that leverages the functionality of „AbpDbContext”.

For this purpose, the „EntityVersionHelper”, ” EntityVersionFactoryResolver” and „BaseEntityVersionFactory” have been implemented.

public override async Task<int> SaveChangesAsync(bool acceptAllChangesOnSuccess, CancellationToken cancellationToken = default)
{
    var entries = ChangeTracker.Entries().ToList();
    var entityVersions = await EntityVersionHelper.CreateVersions(entries);
    if (!entityVersions.IsNullOrEmpty())
    {
        await AddRangeAsync(entityVersions);
        await base.SaveChangesAsync(acceptAllChangesOnSuccess, cancellationToken);
    }
    return await base.SaveChangesAsync(acceptAllChangesOnSuccess, cancellationToken);
}

„EntityVersionHelper” class provides a comprehensive set of tools for managing entity versioning in an application. It includes methods for creating, updating, and filtering version entities, ensuring that only relevant and unique versions are processed. The class relies on various dependencies for logging, ID generation, and data filtering, making it a robust solution for handling entity versioning needs.

The „EntityVersionFactoryResolver” class is essential for managing the lifecycle of version objects in an application. By leveraging a collection of entity version factories and a versioned entity types provider, it ensures that version objects are created and updated appropriately for different entity types. The class encapsulates the logic for selecting the correct factory based on the entity type, facilitating efficient and maintainable versioning processes.

The „BaseEntityVersionFactory” class provides a foundational structure for managing versioned entities within an application. It includes methods for creating and updating version objects, ensuring that only relevant and necessary versioned entities are processed. By extending this base class, developers can create specific implementations for different types of versioned entities, leveraging the provided infrastructure to handle the complexities of entity versioning efficiently.

Every retained copy of the original business entity is saved along with its „EntityVersion”, which stores information about the identifiers of data snapshots (baselines). Meanwhile, the snapshot identifiers are cleared on the modified entity. This way, we obtain an archival record for one or more baselines and a current record (for the current version of the data).

[ExposeServices(typeof(IEntityVersionFactory))] public class SampleVersionFactory : BaseEntityVersionFactory, ITransientDependency { public SampleVersionFactory( IEntityOrginalValuesManager entityOrginalValuesManager) : base(VersionedEntityType.Sample, entityOrginalValuesManager) { }

protected override async Task<IVersionedEntity> GetRootEntity(object entity)
{
    return await GetSourceSample(entity);
}

private async Task<Sample> GetSourceSample(object entity)
{
    if (entity is Sample sample)
    {
        return await Task.FromResult(sample);
    }
    return null;
}

protected override async Task<ICollection<object>> CreateInternal(object entity)
{
    var result = new List<object>();
    var sourceSample = await GetSourceSample(entity);

    if (sourceSample != null)
    {
        var sampleVersion = await EntityOrginalValuesManager.CreateEntityVersion(sourceSample);
        await SetIdentifier(sourceSample, sampleVersion );
        result.Add(sampleVersion );
    }
    return result;
}

private async Task SetIdentifier(Sample entity, Sample newEntity)
{
    var identifierValues = await EntityOrginalValuesManager.GetNavigationPropertyValues(entity, e => e.Identifier);
    var identifier = new SampleIdentifier(
        GetPropertyValue<string>(identifierValues, nameof(entity.Identifier.ProjectKeyPrefix)),
        GetPropertyValue<string>(identifierValues, nameof(entity.Identifier.FullPrefix)),
        GetPropertyValue<string>(identifierValues, nameof(entity.Identifier.FullIdentifier)),
        GetPropertyValue<int>(identifierValues, nameof(entity.Identifier.SequenceId))
        );
    newEntity.SetIdentifier(identifier);
    await EntityOrginalValuesManager.PopulateTenant(newEntity, nameof(newEntity.Identifier), entity.TenantId);
} }

Viewing Baselines

One of the requirements was the ability to view retained baselines (data snapshots) in the same way we use the application with current data. To achieve this, the „CurrentDataVersion” class was introduced.

public class CurrentDataVersion : ICurrentDataVersion, ITransientDependency
{
    public virtual bool IsAvailable => Id.HasValue;
    public virtual Guid? Id => _currentDataVersionAccessor.Current?.Id;
    public DateTime? CreationTime => _currentDataVersionAccessor.Current?.CreationTime;
    ….
}

The „CurrentDataVersion” class encapsulates the logic for managing the current data version within a specific scope. It provides mechanisms to change the current data version and ensures that the previous version can be restored when needed. This is useful for scenarios where the application needs to temporarily switch to a different data version and then revert back.

Users can choose which baseline they want to view while browsing the list of Baselines. The data version identifier from baseline is saved to the User Settings using the ABP.IO Setting Management Module. To be able to set „CurrentDataVersion”, „DataVersionMiddleware” class has been implemented.

public class DataVersionMiddleware : IMiddleware, ITransientDependency
{
    private readonly ICurrentDataVersion _currentDataVersion;
    private readonly IDataVersionManager _dataVersionManager;

    public DataVersionMiddleware(
        ICurrentDataVersion currentDataVersion,
        IDataVersionManager dataVersionManager)
    {
        _currentDataVersion = currentDataVersion;
        _dataVersionManager = dataVersionManager;
    }

    public async Task InvokeAsync(HttpContext context, RequestDelegate next)
    {
        var currentDataVersion = await _dataVersionManager.GetDataVersionSettings();
        using (_currentDataVersion.Change(currentDataVersion.Id, currentDataVersion.DataVersionType, currentDataVersion.CreationTime))
        {
            await next(context);
        }
    }
}

Global Data Filter

The next step was to implement a global data filter for all queries in the application. We utilized Entity Framework Core’s Global Query Filters, encapsulated within the ABP framework’s „AbpDbContext”. This integration allows us to seamlessly apply these filters across all database queries in a unified manner.

if (typeof(IVersionedEntity).IsAssignableFrom(typeof(TEntity)))
{
    Expression<Func<TEntity, bool>> versionedEntityFilter =
        e => !VersionedEntityFilterEnabled ||
            (!CurrentVersionedRecordVersionId.HasValue && ((IVersionedEntity)e).IsCurrentVersionedEntity
                || CurrentVersionedRecordVersionId.HasValue && ((IVersionedEntity)e).Version.VersionedEntityVersionIds.Contains(CurrentVersionedRecordVersionId));
    expression = expression == null
        ? versionedEntityFilter
        : QueryFilterExpressionHelper.CombineExpressions(expression, versionedEntityFilter);
}

Summary

“Baselines” is our response to clients looking for an effective way to manage historical data snapshots. With this system, we can ensure that every entity in the system maintains a detailed record of changes over time, providing clear, stable baselines essential for historical reference and analysis. Additionally, integrating our data versioning system with the ABP.io platform proved to be a relatively simple and effective process. The ABP.io framework offers a rich set of tools and design patterns that facilitate the implementation of our advanced data versioning and snapshot management requirements. Thanks to the modular architecture of ABP.io, we were able to easily introduce our own versioning components and integrate them with the existing mechanisms of the framework, allowing for seamless and rapid implementation. This, in turn, enabled our team to focus on delivering business value, without wasting time on the need to resolve integration issues. As a result, our “Baselines” has become an even more effective and reliable solution, working smoothly with the wider technology ecosystem of ABP.io.