We built Belly quickly. In order to do that, we bundled a lot of different business concerns into the same application. We used Rails (starting on 3.0 and moving up versions until 3.2) because as a web framework, it is amazing and rapid to develop on top of. Our basic infrastructure looked like the image below (leaving out a lot of details that don’t pertain to rest of this post. For more information, just ask – we’re very transparent about how we do what we do.)
Simple enough. A Rails application, backed by a few datastores. In our case it was MySQL and Mongodb. Most of our data was relational, but there was some instances where using a document store made more sense. Our clients (mobile phones and iPads) were served by an API that was part of the Rails app.
With time, the application became more complex because we had more use cases to handle. One of the biggest hassles were the small changes and often deployment. To attempt to tackle this, the homepage (web views) were torn from the main application and started to consume the API as a client. Good news! Less views/less assets/less files/smaller git repo => faster deployment. This is true to a degree, but the complexity and scope of the application is growing.
Breaking Up the Concerns
But how?! Easy as pie if we want to start over and shut down the business for a bit while we re-tool. We can’t do that (and most likely neither can anyone else – which is why this next part matters). We built services.
The idea is simple. Each concern => a different service. One external API to service clients, and internal APIs to talk between services. This sucks the life out of people who think of the speed and simplicity of database joins. That’s ok. Most of the time, the data can be used without so many joins. If you’re not yet familiar with SOA, the rest of this post may seem a bit out of context. For those of you who have thought about it, but aren’t sure where to start, or how to start – keep reading.
New applications are relatively easy to conceptualize. They’re a new service, new datastore, new API. The trick comes when you figure out how to migrate the old data into the SOA paradigm. After all, it is all buried within ActiveRecord objects with relations everywhere in the legacy app. When you need data from the old db, you have two options.
- Make a new service to access data from the old DB
- Reference the old DB from your new service, and include new data
- Start out fresh with a new DB and new service. Copy the data and be on your way.
For most cases method 3, while ideal, isn’t practical. Other parts of your Rails application need access to that data, and often do so directly through the database.
Using method 1 or method 2 doesn’t fully bring most SOA infrastructures to where they want to be in the long term (each service fully backed by its own datastore(s) and supporting infrastructure). It will, however, allow you to build out the software quickly and enable you to focus on separating the data into individual stores at a later time.
The diagrams below illustrates how the 3 methods will look like in your infrastructure. Note that all of the methods are still connected to the API, so that the end users can maintain their applications independent of your infrastructure changes.
Method 1 is making a service with no additional data over what you currently have in the application. You have an external service now, referenced from your old application via HTTP (or another transport). This means that you can still modify business logic in a self-contained application. Ideally, you’ll remove all of the direct references to the database for those models from the old application. (this may eventually lead to the situation described below in Method 2, where you have a service specific datastore in addition to referencing the legacy datastore)
Method 2 will cause a dependency mess going forward, but it is also the fastest option to get your new service out the door. Set up the ORM in your new service to reach into the old db for that model (if you’re using Ruby, this is ActiveRecord or DataMapper most likely). You may also have a new datastore specific to your service. Eventually, you’ll likely want to migrate away from utilizing the old datastore as a shared resource between the old application and your new service
If you have a brand new concern with new logic and new data, consider starting the service as a standalone entity. This way, it can use its own datastore, have its own api and be completely independent from your legacy application. Ideally, this is the goal of having a complete service oriented architecture, where all of the data and logic is bundled with its own concern. In this migration path, you may still have references from your legacy api, or you may choose to open up the new api to the clients (possibly proxying via certain paths at a load balancer)
If you’re end goal is to completely decouple all of your system’s concerns, you’re likely going to end up with a service cloud that resemble the diagram below.
Migrating isn’t going to be pretty, but it certainly is possible without having to completely rewrite everything at the same time.