Uggg, Your Schema Sucks, but I Want Your Data

At Belly, our product code is built in house, but for our CRM we utilize Salesfoce. Lots of companies do. They’re huge, they scale well, and they let teams of people manage their own data. Awesome, right? It is. I’m not knocking it. When non-database engineers manage a database, however, things can quickly spiral out of control. Lack of normalization, lack of data consistency, etc.

Some of the data that we’ve pulled from Salesforce comes in handy for making our product code better. Here’s the rub. We can put logic all over our product code to do custom interactions with Salesforce or we can simply grab the data and store it where we see fit (in our case, we use MySQL for our relational data and Mongo, Hadoop, or the best tool for the job for non-relational data).

We’ve separated the Salesforce logic into it own service. I’m going to use the word Salesforce a lot, so from here on out, I’ll refer to it as SF. That service’s only concern is returning data from SF in a way that makes sense to us and our other services. We can also change how our teams use SF and quickly adapt without changing any of our primary product code.

We have an object in SF called Opportunity. I’m pretty sure it’s one of their standard models. It has fields like the ones below (for the sake of brevity, I won’t include everything).

1
2
StageName Business_ID__c

StageName – not the way I’d camel case a variable, but that’s just an opinion of not starting variables (in this case, attributes) with a capital letter. Whatever. It isn’t terrible.

Business_ID__c – WHAT THE FUCK IS THAT? SF tends to mark custom variables with __c. Cool, but there’s no way I want to ask all of the other pieces of code to use this monstrosity of a convention.

So? We have a map (the rest of this post is written in ruby. Its easy to read, and a pleasure to write):

1
2
3
4
5
6
def map { :stage_name => 'StageName', :business_id => 'Business_ID__c' } end

We keep this inside a class that quickly translates from our variable names to SF ones.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
class Opportunity < OpenStruct def api_attributes output = Hashie::Mash.new map.each do |k,v| output[k] = self.send(k).nil? ? self.send(v) : self.send(k) end output end def salesforce_attributes output = Hashie::Mash.new map.each do |k,v| output[v] = self.send(v).nil? ? self.send(k) : self.send(v) end output end def map { :stage_name => 'StageName', :business_id => 'Business_ID__c' } end end

We call this from API/controller logic below (this example uses Grape).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
class BusinessApi < Grape::API before do @sf_client = Databasedotcom::Client.new( :client_id => ENV["SALESFORCE_CLIENT_ID"], :client_secret => ENV["SALESFORCE_CLIENT_SECRET"], :host => ENV["SALESFORCE_HOST"] ) @sf_client.authenticate( :username => ENV["SALESFORCE_USERNAME"], :password => ENV["SALESFORCE_PASSWORD"] ) @sf_client.sobject_module = Salesforce @sf_client.materialize('Opportunity') end desc 'Return Salesforce details about a Belly Business' params do requires :id, type: String, desc: "(Belly) ID of the business" end route_param :id do get do sf_opp = nil begin sf_opp = Salesforce::Opportunity.find_by_Business_ID__c(params[:id]) error!({code: 'opportunity_not_found', message: 'Opportunity not found'}, 404) unless sf_opp rescue Exception => e return error!({code: 'salesforce_error', message: 'e.to_s'}, 500) end opportunity = Opportunity.new(sf_opp.attributes) return opportunity.api_attributes end end end

The end result is that we can ask for the data for a business using our product code’s ID, and get back a very sane looking result. Also, Belly can easily update our mapping process as we change how SF is used without having to adjust multiple projects to adapt to such changes.

1
curl http://servername/businesses/1000 #=> {'stage_name':'Closed Won', 'business_id': '1000'}

Bang! Your data, the way you want it with minimal mess.

Ask a question or share this article, we’d love to hear from you!

Tweet