Migrating from On-Prem to Cloud

Updated: Sep 7

By Jin Kim


Introduction

Cloud migration is the process of moving some or all digital assets like data, workloads, IT resources, or applications to the cloud.


A well planned migration to the cloud should happen gradually and incrementally until all objectives are reached.


A successful migration should be as seamless as possible to the organization, with no disruption, and minimal downtime and disruption of services.


There are 5 popular cloud migration methods: Rehost, Replatform, Refactor, Rebuild, and Replace.


Among those 5 methods, Rehost and Replatform (Revise) cloud migration strategies result in highest long-term TCO. [Gartner Report “How to Build a Cloud Migration Cost Estimate” - March 2021]


Among 5 of them, Refactor, Rebuild, and Replace may lead to vendor lock-in issues as the more cloud-native your application is, the more cloud features it is likely to consume. This makes applications tightly coupled to the public cloud vendor you are using.


Growing organizations are especially vulnerable to vendor lock-in because the initial solution that worked at first is often no longer sufficient as businesses expands.


By 2025, over 80% of organizations will use more than one cloud service provider (CSP) for their data and analytics use cases, making it critical to prioritize an independent and CSP-neutral integration strategy to avoid vendor lock-ins. [Magic Quadrant for Data Integration Tools, 2021]


Rehost is a relatively low migration effort. The most important benefit of this strategy is migration speed. Moreover, the migration can often be done automatically using a variety of cloud workload mobility tools such as RiverMeadow, CloudEndure, etc.


Replatform is good for organizations for taking advantage of cloud with improved scalability, elasticity, cost and performance, but cannot invest the required resources and effort to re-architect the application. For example, Replatform is used when replacing database backends of applications with a corresponding PaaS database solution of a cloud provider.


Replace is a good strategy where the legacy application is fully replaced by a SaaS-solution that provides the same or similar capabilities. For example, organizations can replace the outdated on-premise CRM software with a SaaS-solution like SalesForce or HubSpot.


For Refactor and Rebuild migration strategies, snapblocs can help organizations to avoid vendor lock-in and lock-up through a catalog of pre-built data platform blueprints using widely adopted community open source components.


snapblocs Cloud Migration Use Cases

snapblocs offers the following data platforms for the data migration that can be used in Refactor, Rebuild migration strategies without a CSP lock-in:

  • Data Flow, a platform for moving data from variety input data sources to target data destinations in stream or bulk mode.

  • Data Transformation, a platform for ​​converting data from one into another structure

  • Data as a Service, a platform for simplifying access, accelerating analytical processing, curating datasets

  • Data Lake, a platform for a set of integrated solutions to ingest data from the multiple data sources into a Data Lake, to orchestrate data transformation and data curation processes

snapblocs also offers the following application platforms for migrating applications that are used in the Refactor, and Rebuild migration methods without a CSP lock-in:

  • Kubernetes+, a platform for developing cloud native portable applications

  • Microservices+, a platform for ​​developing applications as a suite of small loosely-coupled services

snapblocs Benefits

snapblocs helps to reduce risks and increase the velocity of the cloud migration initiative.


Safe Release

As snapblocs provides the configurable UI solution for creating a data platform that eliminates coding, manual configuration for increasing speed, consistency, and reliability.


Each snapblocs platform comes with Clone and Move functions that safely accelerate the release cycle from lower release environment to upper environment.


Once a stack is configured for one project, an authorized user can easily clone the stack configuration within the same project or to a different project, and then update the configuration (such as cluster parameters and target environment settings). Projects can be used to represent different environments or a set related environments. Cloning enables a smooth error free method for managing lower and upper environments. Stacks can also be moved across projects while retaining their original configuration.


Increase Testability

snapblocs allows developers to create data platforms in their testing environment to safely test their new data structure or transformation before releasing to a shared integration environment. An individual data platform for each developer allows them to increase the testability using separate and isolated platform environments.


Improved Quality

snapblocs improves the quality of migration solutions with a more reliable assessment of system quality attributes like performance, security, Observability, interoperability, reliability, availability, and scalability.


Higher productivity

snapblocs allows building migration solutions faster by choosing a suitable architecture from well-designed architecture blueprints instead of designing and building the solutions from scratch.


Better maintainability

The migration solution built with snapblocs is easier to maintain, as snapblocs provides well-designed architecture that is straightforward and easy to understand.


Higher adaptability

As snapblocs provides a well designed software architecture with a clear separation of concerns, snapblocs helps to quickly add new features for supporting new migration requirements. Also, it makes it easy to replace old implementations (or software) with new ones for improving SLA.


Lower TCO

snapblocs reduces the Total Cost of Ownership with faster time to release and lower operation and maintenance costs


No Vendor Lock-In / Multi-Cloud

snapblocs help organizations avoid vendor lock-in and lock-up through a catalog of pre-built data platform blueprints using widely adopted community open source components. snapblocs embraces best in class open-source software (with an optional enterprise support license) that are deployed with a choice of cloud vendors.

Once a migration application is built with snapbloc platforms, the application can be run on major public cloud providers because each snapblocs platform is architectured as a vendor-neutral solution.


Built-in Observability

The snapblocs SaaS-based UI, dpStudio, helps DevOps to monitor and manage the operational health of stacks with built-in Observability metrics such as logs, APM data, and alerts. This enables real time analysis of deployed stacks while gaining actionable insights quickly and securely.

​​

Resume and Pause

snapblocs provides Pause function on any stack without destroying the data or configuration. Pausing a stack saves cloud provider costs and the stack can be deployed again when needed without having to rebuild the stack.


Once a stack is in a paused state, all Kubernetes worker nodes will be released (0 worker nodes) except for the control plane. During the paused state, all EC2 nodes will be terminated and no stack components or business applications will be accessible, as those Kubernetes worker pods were terminated. The storage volumes will remain so that no data will be lost during the paused state. When a stack is resumed those volumes will be attached to newly created worker nodes (running containers for the application, data, and stack components).


Security

Generally speaking, the cloud environment provides better security than an on-prem data center because cloud service providers are required to meet the highest security standards in the industry, set by health, financial, and government institutions. This enables certifications like SOC2, ISO27001, HIPAA, and PCI to be much easier to obtain when running on a computing infrastructure that is already certified.


As snapblocs follows the recommended best practices of cloud computing, the migration solutions built from snapblocs are secure.

For example, all internal and external network communication use TLS secure connections. It also limits the network endpoints on the attack surface by putting all network endpoints inside of private Subnets except public facing APIs. ​​snapblocs provides a fine-grained access control for each user using RBAC-based access control.


General migration strategies from on-prem to cloud

There are 5 popular cloud migration strategies based on the way each company wants to use the cloud to accomplish their goals: Rehosting, Replatforming, Refactoring, Rebuild, Replace.

[The Cloud Strategy Cookbook, 2021 - Gartner report: February 2021]


[How to Build a Cloud Migration Cost Estimate - Gartner report: March 2021]


Rehosting (Lift-and-shift)

This is a brute force, shallow and fast cloud migration method for organizations looking to get out of the data center and stop managing hardware. Lift-and-shift provides the same software that a company used in the data center, but moves it to the cloud. This is to replicate their legacy applications whole and intact onto a cloud platform.


Rehosting doesn’t require much learning curve for the cloud applications except managing new infrastructure and security, since they work exactly the same as before.


However, companies can’t take full advantage of the speed and versatility the cloud can provide with a lift-and-shift migration model as it just moves the code to a new location.


This model is best suited to companies that don't expect unusual high peaks in the transaction volumes and business changes are slow or used as a tactical means to vacate on-prem facilities quickly.


Replatforming (Revise)

It is similar to rehosting in that the code is not altered excessively. However, replatforming incorporates minimum code/configuration changes for the purpose of taking advantage of the new cloud infrastructure such as databases or filesystems.


For example, you may choose to modify the way the app interacts with the database to benefit from automation and a more capable database infrastructure. For instance, when re-platforming to AWS, an on-premises database could be replaced with AWS DynamoDB.


​​Replatforming retains the core application architecture, and cloud adoption enables some level of optimization.


Refactoring (Re-architecture)

It requires re-architecting applications to better support the cloud environment.

This strategy involves modifying your existing software, or a large chunk of the code base, to take advantage of cloud-based features and the flexibility and elasticity that comes with them.


Refactoring is much more complex than other cloud migration approaches, because it requires application code changes, and needs to be tested carefully to avoid regressions in functionality. You also need to ensure that the application is built for better resource utilization in the cloud, to avoid incurring high costs.

While the refactoring approach is the most time-consuming and resource-intensive, it can provide the highest return on investment once you are running in the cloud.


It may introduce Vendor lock-in issues as the more cloud-native your application is, the more cloud features it is likely to consume. This makes applications tightly coupled to the public cloud you are using.


Rebuilding

Rebuilding involves a complete rewrite towards cloud-native architecture because the existing application was created and does not align with the modern-day cloud environment.


​​This is the hardest option to implement, requires the most planning for the future, and comes with the most upfront cost, but it’s the only option that lets you utilize the full strength and flexibility of the cloud. However, for those looking to get into containers or microservice architectures, this is the way to go.


It is a deep cloud integration. You build new applications and take advantage of key cloud capabilities such as scalability, elasticity, robustness, resiliency, redundancy, and higher availability.


It may introduce Vendor lock-in issues similar to a refactoring method as the more cloud-native an application is, the more cloud features it is likely to consume. This makes applications tightly coupled to the public cloud you are using.


Replace

Replace (also called Repurchase) is the strategy where the legacy application is fully replaced by a SaaS-solution that provides the same or similar capabilities.


The migration effort heavily depends on the requirements and options of migrating (live) data. Some SaaS-replacements for on-premise products from the same vendor offer an option to quickly migrate data with little effort or even automatically. Some providers offer analysis tools to assess the to-be-expected migration effort.


Data Migration

Among two cloud migration scopes, the data migration must be initiated before the application migration occurs because an application may need either a partial or full copy of its existing data.


Migrating production data

When and how to migrate the production system data from the legacy on-premises solution to a new cloud version requires careful and detailed planning. Application complexity and architecture, especially the architecture of your data and datastores, will drive the requirements.


There are two common strategies:

  • Do it all at once. Wait until you’ve moved the entire application or service over to the cloud and validated that it works there, and then switch data transactions from the on-premises stack to the cloud stack.

  • Do it a little bit at a time. Move a partial set of data over, test that things are still working, and then move more data until all data is moved. There are many variations of this strategy as well based on business requirements and application architectures and it can become quite complex.


The Do-it-all-at-once strategy may look simpler. However, this is a more risk strategy as it can create a larger downtime during the cut-over transition. It also increases the failure rate due to the complexity of data composition and data structure.


The Do-it-a-little-bit-at-a-time strategy requires more planning for coordinating the incremental dataset release to the new cloud application. Also, it requires much more regression tests for all migrated dataset on the cloud each time when a new cloud version application is released.


This also requires more frequent rebuilding of data platforms for accommodating new data structure or transformation rules as data architecture or transformation rules may be frequently changed.


Data Migration Reality

Data migration generally turns out to be more complex and difficult than first estimated and maintaining quality during the process can be quite difficult. Source data quality often causes a lot of corner case quality problems in the target data set.


Plan for extensive regression and general data quality automation and processes as being essential to any data migration plan. Be prepared to validate the data at every step and also assume some data corner cases will not be found until being used in the application and may not show up right away. Source data sets are seldom without improper records, and the older and more extensive the data set, the more true that is.


Do not underestimate this reality, and any migration project must involve all stakeholders in QA planning, data validation, and final acceptance of the migrated data.


Data Migration Choices

Because no application can run without data, the application must be able to reach data. The location of the data significantly impacts the performance of the applications.


To migrate data to the cloud, it is required to synchronize data, which is the ongoing process of synchronizing data between two or more places and updating changes automatically between them to maintain consistency within systems.


Three Common Ways to Migrate Data to the cloud...


Bi-directional synchronization

Bidirectional synchronization is used when you need to keep data in sync between two systems and allow for active data updates in both systems during any cutover period. In this case both the on-prem and the cloud systems are considered to be live and actively being updated..


To synchronize from one system to another, snapblocs Data Flow platform can be used. The Data Flow stack uses Kafka message bus to guarantee delivery and handle backpressure. To synchronize bi-directionaly, it is recommended to use two Data Flow stacks: one for each direction to independently scale and operate. Also, the Data Flow stack can synchronize data between multi clouds as well.


One way synchronization

Use an on-premises data system with one-way synchronization to a cloud data system, and allow consumers to connect only to the on-premises version. When ready, access to the on-premises version is disabled so that the cloud-based version becomes the main data system, and cloud-based consumers are enabled giving access to the new cloud database.


To synchronize from one system to another, snapblocs Data Flow platform can also be used.


Use a cloud data-migration service

There are many cloud migration services available such as AWS Cloud Migration Service, Azure Database Migration Service, GCP Database Migration Service, and many third party commercial services such as NetApp Cloud Sync


Cloud provider migration services are not generalized, but do offer specialized solutions to migrate databases to their own cloud environments and technology.


File based data can be migrated with solutions such asNetApp Cloud Sync which specializes in handling file based migration such as NFS or CIFS file shares, Amazon S3 object format, Azure Blob, IBM Cloud Object Storage, NetApp StorageGRID® Webscale appliance..


snapblocs Data Flow is a multi-cloud solution and supports over 100+ different data source and destination types.


Migration Steps

When planning a major data migration from on-prem to the cloud, the goal is to ensure everything is done step by step, while minimizing downtime and disruption to services and end users.


The following steps should be considered as part of a successful migration strategy.


Step 1 - Migrate Existing Initial Data

The first step is to create an initial copy of all existing data from on-prem to the cloud.

The data types and format of data structure of destination (cloud) can be different from the source data system. For example, the source can be a RDB and the target destination might be a AWS S3 or MongoDB, etc.


To ensure the correct data conversion, it is recommended to test with a small random sample dataset. To choose the best conversion and destination data system, it is necessary to try many different data systems and conversion strategies.


snapblocs Data Flow platform can be used for the initial trial-and-error/POC phase rebuilding the data pipeline, and transformation is often required to find the best data migration plan. snapblocs enables rapid iteration and rebuilding.


Step 2 - Ongoing Incremental Replication

​​After the completion of initial data migration, the data after the initial data must be continuously migrated to the cloud. This continuous migration can be a near real-time stream replication or scheduled batch replication.


snapblocs Data Flow platform can be used for this step because the Data Flow stack uses Change Data Capture (CDC) that tracks the data that has changed from the on-prem database and relay those changes to the cloud in near real-time via Kafka cluster that provides guarantee delivery solution.


Step 3 - Migrate Less Critical Application First

Choose the first application to migrate to the cloud. The first application shouldn’t be a system that is mission critical to the business or major customer-facing applications. Also, less complex applications can be a good choice to start with.


To validate the migrated data is correct, it is necessary to analyze the migrated dataset on the cloud.


snapblocs Data as a Service (DaaS) platform can be used to easily and quickly analyze the migrated dataset. DaaS provides an aggregated virtual dataset allowing most BI tools simple access to data, accelerating analytical processing and reporting.


Step 4 - Migrate Complex Applications

After completing the migration for less complex applications, the project team will be familiar with new technologies and the migration process. The team is now ready to tackle the more complex and critical applications.


To take full advantage of cloud computing and lower the TCO, it is recommended to refactor (re-architecture) or rebuild applications on the cloud.

[Gartner Report “How to Build a Cloud Migration Cost Estimate” - March 2021]


Kubernetes is recommended for cloud environments.


Kubernetes helps you to build cloud-native microservices-based apps. It also supports containerization of existing apps, thereby becoming the foundation of application modernization enabling faster and more agile development.


snapblocs Kubernetes+ platform can be used to build scalable applications on the cloud.

It has huge additional features on top of the basic Kubernetes such as cost optimization, security, recommended best-practice, Observability, and full lifecycle management.


Also, snapblocs Microservice+ platform can be used to build Microservices applications on the cloud. It is build on top of Kubernetes+ platform. And it comes with additional features for building microservice applications such as API gateway, Kafka message bus, Istio service mesh, kiali observability, and others.


Step 5 - Migrate Legacy ETL processes

While migrating applications and data, many ETL processes must be migrated to the cloud. Due to the Refactoring and Rebuilding on the cloud some legacy ETL processes may no longer be needed. However, if some ETL processes are still required, this may be the last step during the cloud migration.


snapblocs Data Transformation platform can be used for ​​converting data from one into another structure in a real-time stream mode using data pipeline or in a bulk batch mode using ETL.


Application Migration

If you choose either Refactoring or Rebuild method to migrate to the cloud, it is recommended to use Kubernetes and optionally Microservices platforms.


Kubernetes helps you to build cloud-native microservices-based apps. It also supports containerization of existing apps, thereby becoming the foundation of application modernization and letting you develop apps faster.


Kubernetes leverages some form of Platform as a Service (PaaS) when used on the Cloud. It utilises the flexibility of Infrastructure as a Service (IaaS) and enables portability and simplified scaling.

There are many benefits with Kubernetes on the cloud such as:

  • ​​Portability and flexibility

  • Multi-cloud capability

  • Increased developer productivity

  • Open source

However, there are know disadvantages such as:

  • Too complex to provision and manage Kubernetes cluster

  • Hard to find experienced talent

  • Requires DevOps, Self-service

  • Too expensive to run

To overcome this disadvantage, snapblocs provides Kubernetes+ platform and Microservices+ platform for Refactoring and Rebuilding cloud migration methods.


snapblocs Kubernetes+ Platform provides a SaaS-managed service for Kubernetes on cloud provider Kubernetes (EKS on AWS, GKE on GCP, AKS on Azure). This allows a team to use snapblocs dashboard UI to create a Kubernetes stack by configuring the cloud provider and Kubernetes options such as VPC, Subnets, Kubernetes version, and cluster capacity. This allows a project team to focus business applications without managing Kubernetes clusters.


Also, snapblocs dashboard allows a team to manage the lifecycle of the Kubernetes stack (start, update, terminate, pause, resume, clone, etc.) and monitor the runtime behavior of the stack through build-in Observability features that measure how well data is moving and the health of the internal states of the components.

snapblocs Microservice+ Platform provides full lifecycle management of microservice modules on Kubernetes. It provides API gateway, Kafka message bus, Istio service mesh, kiali observability, and others. This allows a team using snapblocs dashboard UI to provision/pause/resume/monitor/teardown microservice on Kubernetes+ platform. The project team can focus on the development of microservice applications without managing the Microservice infrastructure.


Microservice architecture is about decomposing a software system into autonomous modules which are independently deployable, maintainable, and testable. These loosely coupled and independently deployable can communicate via lightweight, language-agnostic ways, and together they fulfill the business goal.


Closing summary

There are many reasons companies are migrating from on-prem to the cloud, such as:

  • Reducing IT costs.

  • Increasing business agility.

  • Improving Security.

  • Eliminating end-of-life concerns.

  • Consolidating data centers.

  • Enabling digital transformation.

  • Accelerating growth.

  • Leveraging new technologies.

There is no magic silver bullet for successfully migrating to the cloud but careful planning and leveraging as many tools and technologies as required can deliver a successful migration.


In the case of Refactor and Rebuild migration strategies, snapblocs can help organizations get the job done much faster with higher quality and scale. In addition the newly created solutions can avoid vendor lock-in and lock-up through a catalog of pre-built data platform blueprints using widely adopted community open source components.


snapblocs offers the following data platforms for the data migration that can be used in Refactor, Rebuild migration strategies without a CSP lock-in:

  • Data Flow, a platform for moving data from variety input data sources to target data destinations in stream or bulk mode.

  • Data Transformation, a platform for ​​converting data from one into another structure.

  • Data as a Service, a platform for simplifying access, accelerating analytical processing, curating datasets.

  • Data Lake, a platform for a set of integrated solutions to ingest data from the multiple data sources into a Data Lake, to orchestrate data transformation and data curation processes.

snapblocs also offers the following application platforms for migrating applications that can be used in the Refactor, and Rebuild migration methods without a CSP lock-in:

  • Kubernetes+, a platform for developing cloud native portable applications

  • Microservices+, a platform for ​​developing applications as a suite of small loosely-coupled services

snapblocs does not help in every data migration use case but for those where it does, the solutions are not only very effective but can be deployed in days instead of months due to the ready-to-go solution blueprint library. The high level of automation and agility offered allows for rapid iteration within the data migration project while lowering implementation costs significantly.


Reference


Create a free developer account. No credit card required.

OR create a Trial team account, with a 14 day free trial and pay for what you use after the trial. No credit card required during the free trial.

Both accounts come with a free Sandbox credits so you can run, test, and deploy blueprint stacks.


You can also book a free 1:1 demo with our team.



39 views