SpinSys is an Amazon Technology Partner with AWS Certified Professionals specializing in managing big data in the cloud. Our expertise includes cloud migrations, legacy applications migration and modernization, master data management, cloud infrastructure management, and cost optimizations for government and commercial enterprises.
SpinSys supports one of the largest military health care systems running workloads on AWS. As part of the enterprise migration and consolidations for efficiency of operations in the cloud, SpinSys, in partnership with AWS, was tasked to support the enterprise consolidation of legacy databases; the transition from on-premises to the cloud is a major component of the enterprise digital transformation process.
The SpinSys Enterprise Data Warehouse team and AWS established a robust environment that included an Amazon Redshift cluster as the enterprise Data Warehouse in the cloud in September 2021. This environment has been steadily growing in support of billions of transactions on a daily basis, scaling to meet the customer needs. However, an enterprise data mart containing over 25 years of historical data remained hosted in an on-premises data center and was not part of the original data consolidation plan. The customer requested our team co-locate these two data warehouses into one Amazon Redshift cluster within their AWS Virtual Private Cloud (VPC) to provide a streamlined, unified solution as the customer’s largest health data repository for analysis and research.
Data Warehouse Migration and Consolidation
The SpinSys data migration team and AWS teams were tasked to execute a Well Architected Review with a primary task to migrate the legacy data from an on-premises data center to the AWS-hosted / SpinSys-managed environment. The on-prem data repository was a complex data mart designed for easy self-service access to 4.5 Terabytes (TBs) of enterprise data for current and former beneficiaries. In addition, the on-prem environment provided powerful ad-hoc query tools used to manage and oversee operations worldwide for over 2,500 users, supporting an average of 1,500 queries daily and surges doubling to 3,000 during peak demand.
We leveraged our cloud migration expertise to migrate data schemas and stored procedures from the DB2 database into Redshift and migrate the historic data. Our team automated the Extract, Transform, Load (ETL) process for more than 600 fiscal year tables. These nearly 7 billion rows/records of Protected Health Information/Personally Identifiable Information (PHI/PII) data were migrated to the Amazon Redshift cluster alongside existing data warehouse data. We leveraged SpinSys MDACA products to facilitate integration with Redshift and enhance the end user experience for the more than 100K queries in the first 4 months since migration.
The SpinSys team conducted Limited User Testing with super users from the functional community to confirm data availability for use within authorized query and analysis tools of choice. Once testing confirmed the new environment was ready for mainstream use, we conducted extensive User Acceptance Testing with the entire user community. This was essential to ensure all queries returned expected results and to instill confidence in the end-to-end technical solution for the enterprise users.
With close stakeholder engagement and transparent communications, the DB2 to AWS Redshift migration matched the predecessor Data Warehouse Migration success launching Amazon Redshift into the customer’s VPC. The migration was completed with a seamless production cutover without any downtime for end users resulting in the first-ever consolidation of data warehouses for the customer and workloads running on AWS GovCloud. The DB2 to Redshift Migration is projected to save our customer $2.5M over the next five years. Furthermore, our customer’s analysts and researchers will benefit from the security, scalability, and availability of Amazon Redshift within the AWS well-architected framework.
Lou Ferrucci
Healthcare Innovation Director
SpinSys
Faaruwq Muhammad
Project Manager
SpinSys
