Case Study: Rolling Back Changes in a CI/CD Pipeline

The Scenario

Imagine a situation where a development team has recently added a new feature that required changes to the database schema. The feature has been deployed to production but soon after, issues start appearing. Immediate action is needed, and the team decides to roll back the database changes.

For this case study, let’s assume the team is using Azure DevOps for their CI/CD pipeline and Liquibase for database migration.

Steps for Rolling Back

1. Identify the ChangeSet

The first step in rolling back is to identify the changeSet that needs to be reverted. Liquibase maintains a history of applied changes, making it easier to pinpoint the specific changeSet.

2. Prepare the Rollback Script

Liquibase allows you to specify rollback logic within the same changeSet. This logic could be something like:

<changeSet id="3" author="dev">
    <createTable tableName="new_feature_table">
        <!-- Table definition here -->
    </createTable>
    <rollback>
        <dropTable tableName="new_feature_table"/>
    </rollback>
</changeSet>Code language: HTML, XML (xml)

3. Local Testing

Before triggering the rollback in production, it’s crucial to test it in a local environment to ensure it behaves as expected.

4. Update CI/CD Configuration

The Azure DevOps pipeline is configured to run a step that executes the Liquibase rollback command, specifying the changeSet to roll back to.

---

CI/CD Pipeline Integration

Azure DevOps Configuration

1. Source Code Management: The pipeline fetches the latest code and database migration scripts.
2. Pre-Rollback Tests: Automated tests are run to simulate the rollback and catch any potential issues.
3. Liquibase Rollback Step: A script task is added to the YAML pipeline to perform the rollback using Liquibase.
4. Monitoring and Alerting: Real-time monitoring is enabled to watch for any issues during the rollback process.
5. Audit and Logs: All rollback activities are logged for auditing and debugging purposes.

---

Post-Rollback Steps

1. Verification: After the rollback is complete, further tests are run to confirm that the database is in the expected state.
2. Communication: The team, stakeholders, and (if necessary) the users are informed about the rollback and the subsequent actions to fix the issues.
3. Root Cause Analysis: A thorough analysis is conducted to determine the cause of the issue and to implement safeguards against similar issues in the future.

---

This case study outlines the steps and considerations for effectively rolling back database changes in a CI/CD pipeline using Azure DevOps and Liquibase. From identifying the changeSet to preparing and testing the rollback script, each step is geared toward minimizing disruption and ensuring a smooth return to a stable state.