Salesforce and DevOps Part 2a - Dev Tools

After rambling about my views for a while in the start of this series, I thought I'd take it down a couple of levels and talk about the tools that I've used or know about that are compatible with Salesforce and DevOps.

This will be a long post, so I divided it in 2, one for Dev, one for Ops. But please bear with me, as it's important and will be needed for the next posts in the series and it can be useful for people as a go-to list. In that spirit, please send me info in the comments about other tools that I missed and you use and love, and I'll keep updating after checking them out.

I'll break these 2 posts down into a few categories, as much for ease of reading as for highlighting the importance of each and the width that DevOps brings into any engagement.

Of course, this is not a definitive list, as there are a lot of tools that do super cool things, but either I haven't used them, or they don't apply to Salesforce (usually because they're infrastructure related - Salesforce is fully managed - and I'll probably cover a few of those in a future post about DevOps in general, outside of only Salesforce, stay tuned), or both. For more information on the set of tools that exist out there and are widely used, check out the periodic table of DevOps, for example.

Note: this post and the ones that follow will be related to Salesforce's core platform, not touching on other areas like Marketing cloud, B2C commerce cloud (what was Demandware), Heroku and so on.

Ok, let's get straight to it.

As I mentioned lightly in my previous post, Salesforce has the metadata API to move changes from one place to the other, and it also has the Force.com migration tool (basically a way to use ant with the metadata API). These tools work for what they do, but are limited, as the API doesn't support all the metadata needed for complete automated deployments, and some of the areas it supports are messy, buggy or both (profiles anyone?).

Salesforce then came up with the Tooling API, faster and for different things than the metadata API, but it allowed people to build custom development tools. This API added to the ecosystem, but it was still lacking in completeness.

As an initiative to improve the developer experience and give more flexibility and freedom to move into source driven development (instead of "just winging it" as I feel it is otherwise), Salesforce came up with a CLI and Salesforce DX. They're still in early stages and are limited in some areas, but I'm not going to get into a lot of detail in this post about this, as I'll write a full post on DX in this series. I point this out because it affects the way you use some of the tools below.

IDE

This is a contentious topic, as everyone has a different view and preferred choices, but I promise I won't get into arguments or choose favourites (as much as I can).

For some context, Salesforce has supported a version of the Force.com IDE on eclipse for years, and love it or hate it, there was a need for more options, as it definitely has had some problems over the years. So MavensMate got popular, which continues to be widely used to this day, but it's now officially unsupported, meaning it won't get updates anymore. This is a big problem, given that Salesforce updates itself every 4 months or so.

This brings me to the list of some of the most popular IDEs that are out there:

• The developer console: Salesforce's cloud editor (and a bit more) it's OK, but not really relevant in this post, as it doesn't apply to source driven development
• Force.com IDE: this one is free, still exists and it's still supported for the "ant" version of development. Doesn't support DX (there was a version that started, but they killed it in favour of VS code)
• VS Code with Salesforce extensions plugin: this is a lightweight IDE, free to use and it's the one that Salesforce is pushing and supporting actively at the moment
• MavensMate: this is not an IDE, it's a plugin. It's free to use but, as I mentioned previously, it's not being worked on anymore. I've seen it used with the following editors:
• SublimeText: paid, but light and pretty flexible
• Atom: free and lightweight, this one is popular, though I haven't used it too much
• VS code: as above
• Welkin suite: it's a paid, full proprietary solution. Check it out here. This is actively supported.
• Illuminated Cloud: this one is also a plugin (a paid one). It can be used with Jetbrains' IntelliJ IDEA and Webstorm IDEs. Check it out here.

IDE Verdict:

In my opinion, if you're able to use only DX (lucky you!), I'd go with VS Code with Salesforce extensions. It's free, lightweight, integrated with CLI and Git and it's officially supported by Salesforce. 

If you're not going with DX (or even if you are, as it's supported) and have the money, I'd go with Illuminated Cloud. I haven't decided on top of which editor, but I'll update this when I dig in a little more on that.

Source control

The most important part of the whole DevOps "thing", if it's not in source control it doesn't exist!

Even though there are many different systems for source control, I'm going to be only talking about Git, which is probably the de facto standard for source control. As it's distributed, it makes it great for collaboration and coordination. Check out the wikipedia page here for more info.

For git, you'll need clients and a server, below the list of the ones I've been in touch with:

Clients:

• Git CLI: it's free, it has everything, of course, it's the foundation for the use of git and, as most CLIs tend to do, they make you feel pretty good in your matrix like terminal, if you're that kind of dev
• Atlassian's Sourcetree, Github desktop or similar: these are mostly free, standalone desktop apps, which give you nice buttons and visual aids, but in the background, they're of course using the CLI. If you don't like typing commands and like the nice and flashy buttons and stuff, these are for you
• IDE integration: most IDEs have a git integration, which lets you interact with source control directly from it, without the need of another app or even the terminal. These are quite handy and I find myself using these more and more often.

All in all, it's a personal choice, the options are out there.

Servers:

Again, there are many, but I'll name the most popular ones, at least in my opinion. Also, they're the ones I've worked with, so are the ones I know.

• Atlassian's Bitbucket: pretty complete, fully featured, with a free tier and great integration features
• Github: the most popular one, I think. It's pretty good, fully featured and great integration features. It has a free tier for public repos only
• Gitlab: this is the one I know the least about, but I haven't seen any big downfalls either. It also has a free tier

Verdict:

On clients, it's a dev's choice. I'd just say that in each team, most people should use the same tool, or at least be very careful on how the configuration of things like line breaks and empty spaces are the same, as you could get into a mess of seeing changes that aren't changes and other super fun situations.

On servers, they're all pretty good, to be honest, and it will be a choice of how your full DevOps stack works together. For example, if you're full into the Atlassian suite, then use Bitbucket, as it will integrate nicely with everything else. If not, probably check Github out. The most important part of this is that the server supports Pull or Merge requests and the ability to collaborate on them (this will be heavily used on the next post about processes).

So, choose accordingly, but definitely don't forget to use it.

Artifact repository management

In order to better tag and manage releases, you should probably think of creating artifacts that are inmutable and versioned (I'll explain a bit more on the processes post). I haven't used many repository managers, but I'll name a few here:

• Atlassian Bamboo: the only one I've really used, Bamboo (more on it in the next section) has a feature to create your artifacts based on builds and manage your releases and pipelines. It's quite cool, simple to use and has worked out for me in the past
• JFrog artifactory: fully featured, it's the most well known, as far as I know, but I don't have a lot of info for it
• Sonatype Nexus: fully featured as well, but I don't have a lot of info for it

Verdict:

I can't really say too much about the others, but if you're using Bamboo, think of using your artifacts and releases with deployment projects, as they call it.

Continuous integration / delivery (CI/CD)

This is another area where there are many options and can be contentious, so I'll do my best to stay neutral and just provide the info on the ones I've worked with / played around.

• Jenkins: fully featured, free and open source, with lots of plugins and extensions. You can run it locally, you can host it on your cloud or a public cloud, and you can even get it as SaaS with CloudBees. I think this is the most widely used and flexible one, but, personally, it can feel a bit older as well
• Bitbucket pipelines: at the date of writing this post, I feel that it's a bit limited compared to the other tools that are more feature rich, but it has a free tier, it's yaml based, inside of Bitbucket cloud (SaaS) and can use docker images as builders, which makes it nice and flexible. For bigger teams with a lot of build activity, the free tier doesn't quite work, but for small teams, it might just be the tool for you
• Atlassian Bamboo: now, I usually get the question: wasn't this discontinued? The answer is yes and no, there was a cloud version that got discontinued in lieu of pipelines (I assume), but there's an "on-prem" version that you can download and install on your servers, cloud, etc. with an evaluation period and a very limited "cheap" tier. After that, it gets pretty expensive. Having said that, it's very feature rich, integrates amazingly well with Atlassian's suite (of course) and it has a feature that I haven't quite found in any of the other tools, which is the way it handles branches. Also, you can use docker images as builders, isolating builds and probably saving money on those remote agents that make it so expensive, if you have enough capacity in the server where you host it
• TeamCity: similar to the ones above, feature rich, but there's no SaaS version, that I know of
• CircleCI: this is the popular kid these days, for what I hear. I have to say I haven't delved too deeply into it, but it seems pretty cool, it's completely on the cloud as SaaS. It's yaml based, it has a free tier that might work for smaller teams and it can use docker images as builders, which makes it interesting and flexible
• Copado, Autorabit, Gearset and similar: these tools are a bit weird to categorise, so I put them in this section. These paid tools take over the whole experience and the pipeline, and are quite helpful for teams with less technical knowledge and companies where there's lack of support to manage a stack of different tools. They have good features, but I haven't used them too much, as they tend to be quite closed and not as flexible as I'd like. Feel free to check them out, though, as they have really nice features and are easy to use
• Heroku pipelines: Salesforce's own, there are a few articles on it online, and some buttons and buildpacks by Wade Wegner, but I haven't used it for core Salesforce and seems to me like forcing the use of Heroku just for this, but again, that's my opinion
• Check the periodic table for others, as I don't have much info on the rest

Verdict:

It's one of those tough ones, but same as source control, the choice is probably up to how they fit in your full stack (think of the Atlassian suite decision above). Now, these can get pretty expensive, depending on the size of your team.

If you want to run it for free I'd say take a look at CircleCI first (Bitbucket Pipelines if you're using Bitbucket) and check if they fit for your use case. Then, as your budget needs to grow, move to Jenkins (it can be free, but you have to host it, which costs some - or a lot of - money, depending how you do it), CloudBees, Bamboo and so on.

Also, take a look Copado, Gearset, AutoRabit and the like, for less flexible and robust, but more inclusive experiences. Again, this is my opinion, on the day I'm writing this post.

Code quality review

There are a few of these out there, although not a lot, as Salesforce is a bit of a silo with this topic and the historical lack of tech support I mentioned in my previous post. I've used the following ones for Apex:

• SonarQube with CodeScan plugin: pretty good, paid solution, flexible as it comes with a set of rules that you can customise and add to. It has the downfall that you have to host the SonarQube server and manage that yourself (although there seems to be a cloud version, but I'm not familiar with it).
• Clayton: really nice tool founded by Lorenzo Frattini. It's a paid solution, but it's fully SaaS, integrates well with Bitbucket and Github and it's continuously growing. I may be biased, as I usually can just talk to Lorenzo and get things fast :D. Clayton also does some basic javascript linting as well as basic metadata reviews. It will keep adding features and get more flexible as time goes by.

For lightning component development (i.e. javascript, in this topic), the CLI comes with a lint command that you can use with your builds as you see fit, it works OK. I haven't delved too deeply on this, but I assume you can use any javascript linter, if it fits your needs. 

Verdict:

I'll give a shout out to Lorenzo's Clayton on this one, check it out and hopefully it fits in your flow as it does on mine. Check it out here.

Automated testing

This is a tricky subject, as automated testing is not that popular on the Salesforce platform. The problem, I think, has been that, as Salesforce gets updated every few months, some names could change, some conventions got messed up and then all scripts would fail. 

This has made it too time consuming and not worth it in some cases. Also, there's a lack of technical support for test automation (surprise!). Salesforce devs don't necessarily know how to write test scripts and it turns into a vicious circle. I'm not going into the commercials, but think of hiring a dev just to write these scripts on top of the considerations I mentioned in my views, and you'll see how it's harder to sell automated testing in the Salesforce world.

However, you should definitely automate testing, of course! Plan your budget accordingly and keep all considerations in mind. I'll get a bit more detailed on where to use automated tests when I write my post about the processes.

The usual suspects come to mind for tests automation:

• Selenium
• Cucumber
• Mocha
• Jasmine
• Any other...

Salesforce is inherently a web site, so you can use any of that, if you can and know how to code it and maintain it. 

For running tests in the cloud (you'll need this, as you'll need to simulate environments, connect to your CI tools and so on), there's SauceLabs (I'm sure there are others, but I'm not familiar with them), which I find really cool and integrates very well with DevOps flows.

There's also Provar, which is a Salesforce specific tool, which helps bypass most of Salesforce's peculiarities, so check it out here.

Verdict:

Automate your tests! Sell that and please let me know how you got along with it.

I've used Selenium, Cucumber, SauceLabs and started playing with Mocha for more javascript focused stuff. Those are pretty good, check them out.

Ok, time for a break now. This post will be continued in part 2b, with the Ops side of the tools.