Easy Enterprise Org SFDX and CLI

Auntie Pat Tern started taking Step Dancing classes.  "Break it down", is all she said.

Your new tools can coexist with existing User Interface options.

So I decided to break down my last post: Easy SFDX for Enterprise and Others to prove just how easy it is with 6 simple steps for application lifecycle management with SFDX for the Enterprise.

The Command Line Interface (CLI) for SFDX can be accessed via menu options in Visual Studio Code, or through the terminal, or through a Windows Command Prompt. So I'm going to use the CLI commands in the steps for the most clarity.  And I'm assuming you've enabled the Dev Hub from your Production Org via the Setup options in the Salesforce UI.

Step 1

SFDX force:org:list
Shows which orgs you are connected. You want to connect to your sandbox orgs to manage your development lifecycle, for example: developer, integration, staging.

Step 2

SFDX force:auth:web:login -r https://test.salesforce.com -a orgAlias -s
Connect a sandbox where you will find and update source code and configuration. 
Text in italics will need to be specified by you when you run, and should apply to your work.
-r is followed by the login URL for the org, using test subdomain for sandboxes and login subdomain for your Production org. Be very careful when connecting to Production, test changes before you deploy them, of course.
-a is followed by an alias you are giving to this org connection.  Something like 'Dev' for a developer sandbox maybe? 'Int' for an integration sandbox. You see where I'm going here.
-s indicates that this should be the default org and username for your current work and is important to remember. You can also check the force:org:list to make sure you know which org you are working in at all times.

Step 3

SFDX force:mdapi:retrieve -p somePackage -r r### -u orgAlias -s
Bring code and configuration from your org down to your local computer in a package.xml file. Text in italics will need to be specified by you when you run, and should apply to your work.
-p is followed by a package name, either outbound change set or package defined inside a Salesforce org. 
-r is followed by the directory name on your local computer where the package will be saved.
-u is followed by the alias for the connected org where your package/outbound change set was defined.
 -s indicates a single package structure for the xml file.

The code and configuration should be defined in your existing org using either Setup>Package Manager to automatically include all dependencies and get a ton of related metadata or using Setup>Outbound Change Set just to get the specific code and/or metadata that is changing.  

Packages are super helpful and can be pre-defined in Production so that it is easy to see the full extent of code and customization dependencies related to your custom features. These pre-defined packages carry over to sandboxes you create for super easy org management. Don't upload it, we just want the definition of what it contains, and we want to keep that open and changeable. 

Creating and retrieving an outbound change set in the sandbox where you are starting development gives you a more manageable set of metadata and code, just the bits you need right now. Keep the change set open, don't define any deployment connections, we just want the definition of what it contains and we want to be able to add and remove if necessary.

And of course you can create code locally with commands like force:lightning:component:create and save or deploy those creations to your development sandbox for testing.

I should mention that these outbound change sets and in-org package definitions are all available with Workbench for retrieve and deploy as well, just in case your admins and devs love a familiar UI.

Step 4

Unzip
Using your favorite tools, unzip the "unpackaged.zip" file you just saved to your specified local directory.  Maybe you use a favorite application for zipping and unzipping, maybe jar xf from the command prompt. 

You can now work with this code and configuration in any local app you choose, maybe VS Code. Make sure to save your changes back to your sandbox org. Run automated tests in the org using your favorite tools, check out Setup>Custom Code>Execute Tests.

Step 5

Backup with version control
Save to your favorite repository for backup and version control. You can run git commands from the command prompt or you can literally just drag and drop with cloud-based interfaces for GitHub and BitBucket. You can also connect tools like VS Code to your repository to help you do this automatically.

Step 6

SFDX force:mdapi:deploy -c -d r### -u orgAlias -w 100 -l RunSpecifiedTests -r testa, testb
Deploy changes to the next sandbox as you move your work through the various lifecycle stages. Text in italics will need to be specified by you when you run, and should apply to your work.
-c optionally check the deployment before removing this flag and deploying for keeps.
-d is followed by the local directory name where you have your work and package file (hey, it was '-r' before, but now it is not). You can even do a quick retrieve from sandbox to make sure you are deploying the latest version you tested in the sandbox org.
-u is followed by the alias you have given to the destination org you set up previously.  Not the org you made changes in and tested in, but the org you are deploying changes to for further testing.
-w is followed by a wait time and is used for especially long running deployments.
-l RunSpecifiedTests optionally lets you exclude long running or unrelated tests, otherwise expect all your tests to run in the destination org, even bulk tests. Combine this with the -r option below:
-r is followed by a list of test classes.  Keep your bulk tests in their own classes and separate from unit tests so you don't have to run them during production deployments if you don't have to, for example.

Done. You can look at the Deployment Status right in the destination org, using Setup options you already know.

Four super helpful SFDX CLI commands can really get help you break down your development lifecycle and use new tools where they are most helpful while keeping all the familiar Saleforce UI options to support your orgs and your users and to maintain those valuable audit records for changes in your Enterprise org.

Easy SFDX for Enterprise and Others

When cousin Tim Toady took the doorknob and lock apart on the back door after his key broke, he had a lot of trouble figuring out how to put it back together.  With a hundred tiny pieces laid out across the kitchen, Auntie Pat Tern sifted out a dozen of the most important, handed it to cousin Tim and told him to just use that for now.  Unsatisfied, Tim tried to put a few more parts back in, triggered the spring and the entire thing exploded across the kitchen again.That's when Auntie Pat Tern reminded him of two guiding principles: 1) Keep It So Simple and 2) Keep track of all your changes.  He could have taped each piece to a sheet of paper, in order, as he dismantled the lock, for example.

That reminded me of the guiding principles behind implementing Salesforce Developer Experience for the Enterprise Org.  Staying Agile means finding the Minimum Viable Product for tools you implement, not just products you build, so here is my guide to easy SFDX for the Enterprise. (See part 2 of this post for the actual commands.)

SFDX MVP

1. Discover:  A little reading reveals that SFDX is intended to "enable source-driven development, team collaboration with governance, and new levels of agility".  To accomplish this, it offers the important benefit of using "developer tools you already know." Use what you know, that's easy.

One thing we know in our Enterprise orgs is that configuration changes may happen in Production to meet immediate release requirements.  So the Production org itself is always the absolute source of truth for the org.  You can't just implement source control and claim that the source of truth is there because the bugs your users report will be found in Production and hopefully easily reproduced in Sandbox, that easily created org copy that is a developer tool you already know.

So the first step is to use the developer tools you already know: Production is source of truth for code and metadata and Sandboxes provide great individual and team test environments.  Keep them all. Use Dev Hub in Production to enable command line logins to these orgs.

2. Develop: SFDX includes a Command Line Interface (CLI) that lets you connect to your Enterprise Production org as well as your Sandboxes using simple and batchable commands. That gives you the power to make use of local tools, including local code editing and version control by connecting the CLI to your orgs and pulling down the code to edit locally.

Different Developer Sandboxes and different bits of code on the local computer make keeping track of all the versions and making sure they are merged and integrated into the larger system for testing means that version control is more important than ever.

For version control, we use Git on the local computer and cloud-based systems like GitHub for team backups and version control. Our local git directory can be connected to our own personal branch (or maybe to a feature branch) of the org repository in GitHub. Following tech review, developer branches can be merged into the main code branch. And this can be automated so that a developer's work gets backed up from their local computers to the cloud repo on a regular basis.

You already know about backups and version control if you use something like Microsoft Word.  It periodically auto-saves the working version of your documents. If your computer crashes, you have a choice between loading a version you saved or a version that Word automatically saved.  Version control to the rescue!  That can definitely be useful when you are doing Test Driven Development and your test fails because of your last set of changes and you decide to roll those back to the previous successful version.

If you use Git for version control of your work on your computer, you can keep track of versions.  And you can post those versions to GitHub, BitBucket or similar.  These tools can make team collaboration easier with everyone checking their latest working version into the repository in the cloud. It's like the difference between Word on your local computer and Quip Documents in the cloud.  Collaboration can be simplified with cloud solutions. And you can create a process for branches that get merged into a single main repository based on your own team's organization.

We have different repositories for different platforms we develop on.  And developers are free to create their own repositories for their own organization as well.  One developer keeps every feature in separate branches and another developer keeps just a local repo and a cloud repo, no branches. For our org-wide repository, we have branches based on releases so that short-term and long-term projects are kept separate.

The CLI adds flexibility to our Development Cycle by letting individuals choose tools they are familiar with and develop code locally or continue to develop in the Dev Console even, either way we can create a single shared backup and version control repository of everyone's work thanks to simple CLI commands.

3. Retrieve and Deploy: Following our SFDX process of using what you already know, we can easily coordinate the efforts of admins who make configuration changes in Sandbox into our version control as well as developers who want to edit code with apps they run on their own computer.  Salesforce provides two easy tools to work with the CLI: Packages and Outbound Change Sets.  If you haven't tried both, take a look in Setup>Home for them.

Packages give you the chance to define related sets of custom code and custom configuration metadata. Choose a bit of code and dependency data like custom objects and fields are automatically added. You can create Packages to look for overlap of functional dependencies in your org. And to help you document feature specifications.

Outbound Change Sets give you a bit more control to stick to only code and configuration that has immediately changed, and not include more stable code and configuration from prior development that is already in Production. You can use these to get the most up-to-date configuration changes on the fly from production for anything you need to work on.

Both Packages and Outbound Change Sets can be pulled out of the org and onto the local computer using the CLI retrieve command. These can also be deployed to any other org (yes, including Production) using the CLI deploy command.  Retrieve pulls down a Zip file including a package.xml defining what has been retrieved and folders of metadata, all the same for packages or change sets. You can unzip this into your local Git directory to update your version and you can use the CLI to deploy back to any org after you've made changes. In between, backup ongoing work to shared repositories to merge changes for the next stage of the development cycle.

Sticking with these familiar tools, we not only keep it simple, but we ensure that Admins and Devs can work well together because the org remains the source of truth and work efforts can be tracked, changes can be audited all in the org as usual.

4. Iterate: Since we have the ability to connect to our familiar Enterprise orgs as well as the ability to work on changes locally and use tools for searching and comparing code and configuration versions, we create a stronger collaboration.  Since we can use Git commands on our local computer to manage the backup and version control, we have even better insights along the development cycle, regardless of Sandbox refreshes.

With changes happening in a variety of orgs, local repositories may need to rebase if a branch gets too far our of sync. For Enterprise orgs, the source of truth is what your users see when they report bugs and praise new features. So bringing your local repository back into sync with upstream changes is simplified.

If you have separate repositories for each development stage from Integration to UAT, Staging, and Production, developers can retrieve changes at each point in the process. This makes it even easier to compare versions when a feature breaks after an upstream deployment.  Verifying whether the correct version was deployed with all of its component parts can be done locally, in shared repositories, and by auditing recent deployments right in the orgs.  And reverting to another version of a feature is even easier when you have backups in version control at each step.

Benefits of Change

SFDX helps Enterprise orgs follow Auntie Pat Tern's advice: Keep It So Simple and keep track of all your changes.  By allowing admins and devs both to work with tools most familiar to them, at the same time, SFDX offers greater insight and transparency into the development cycle. Because the CLI opens up opportunities to make use of tools like VS Code and online repositories, debugging throughout the release processes may be simplified as well.  And the CLI gives the power to make everything more automated for a more continuous integration of changes into each new environment at every stage of Enterprise development.

Ultimately, the best implementation of the Salesforce Developer Experience is to make use of what you know and implement it your way, just like the SFDX Developer Guide recommends. Give yourself even more power with command line deployment and retrieval of packages pre-defined in your orgs through easy and familiar Salesforce Setup options. Get even more peace of mind with backups at each stage of the release process thanks to the CLI and Git local repositories connected to shared repositories online.

The Eisenhower Matrix Fallacy

Road trips reveal truths and clarify ideas, at least that's what my Auntie Pat Tern insists. On a recent trip with her and cousin Tim Toady,  this came up when Tim requested a bathroom stop. What he said was, "I could use a bathroom some time soon." And as we flew past exit after exit, Auntie Pat Tern said he should let her know if it becomes urgent, otherwise she wanted to find a good spot to stop for lunch too. That got me thinking about competing interests when it comes to programming and the Fallacy of the Eisenhower Matrix.

Jastrow, J. (1899). Popular Science Monthly US public domain
Duck or Rabbit? It may depend on how long
you allow yourself to analyze the problem.

In a speech at Northwestern University, President Dwight D. Eisenhower said "I have two kinds of problems, the urgent and the important. The urgent are not important, and the important are never urgent. "   

Prioritizing Competing Requirements

Important issues and urgent ones often compete. On our road trip, Auntie Pat Tern knew that a nice place to stop for lunch would also have a nice bathroom, that was part of her selection criteria. She also wanted to get as far as possible to her ultimate goal before the end of the day. Stopping both for lunch and for a separate bathroom pit-stop would delay her from reaching her ultimate goal. So she employed the Eisenhower statement as it was intended, to express that a thing cannot be both urgent and important.

Unfortunately, a lot of people in the business world have worked Eisenhower's statement into a matrix, which suggests that requirements can be both urgent and important.  Trying to accomplish a task as both important and urgent makes no sense. If it is important enough to need an optimal solution, then take the time to discover that optimal solution. If it is urgent enough to resolve immediately, do your best with what you have right now.

Importance means you have the time to be thoughtful and take time to discover the most optimal solution; it is the polar opposite of urgency.

As long as stopping for a bathroom break was important, it was worth waiting for the best location to stop both for food, which was also important, and for clean restrooms. But as soon as the need for a bathroom became urgent, the competing priority for food could be disregarded, at least to some extent. And with greater urgency, even the competing priority for a clean restroom might be set aside; the greatest urgency means you act now even if you have to pull off the road at that tree up ahead.

Eisenhower and Software Architecture 

For software architecture, the urgent and the important need to be kept separate because the important parts of the project should inevitably be the most stable code in the system. The least stable code is the stuff you write out of urgency, when things are likely to change, but we can't wait until we understand all the factors that cause change.

Don't believe the Eisenhower Matrix, it is a fallacy. Important and Urgent are like North and South ends of a magnet. Anything that is equally Important and Urgent lies in the middle, and that part of the magnet is neutral. The neutral zone is where we find the mindless and repetitive tasks that only become urgent when they are neglected for too long and only become important when something out of the ordinary arises.

For Tim Toady on our road trip, he appreciated waiting for a sandwich on Dutch Crunch and a clean restroom because he values good food; it's important as long as he doesn't have more urgent needs.

Why Coders Use Interfaces

Going on a road trip with Auntie Pat Tern and cousin Tim Toady reveals a lot about them as only a road trip can. Tim was called a "whipper snapper" on more than one occasion for criticizing Auntie Pat Tern's driving. "I cannot wait for self-driving cars," he proclaimed while Auntie Pat Tern praised the freedom of the open road and the control she feels while driving a car. Which got me thinking about why interfaces are so important as part of my programming team's design decisions.

Tim Toady prefers to drive arcade games while Auntie Pat Tern craves a 64 1/2 mustang convertible. But they both drive reasonably well in the electric car they have at home and the gas guzzler we rented for our long drive. That's because the interface of gear shift, accelerator, brake, and steering wheel are universally well understood by users.

Accelerate, brake, steer, and shift.
The simplicity of a good interface.

That same interface translates inputs to a multitude of different systems from arcade games to tractor-trailer trucks. When it comes to deciding whether to accelerate or decelerate, the driver/user does not care if the vehicle is electric or gas-powered. A lot of people who drive electric cars even still say "gas pedal".

Autonomous driving vehicles use the same interface as people: shift, accelerate, brake, steer. By using the same interface, a person can take over in an emergency, or maybe just for the fun of driving.

Developers use good interfaces like this to future-proof their code. If the business rules are protected from the mechanism for inputting data thanks to a reliable interface, we can make sure the most important business rules continue to function as expected.

Sure, sometimes the interface has to change a bit with new business automations. When cars got automatic transmissions, the gear shift changed and we lost the clutch. The same happens with code, new automations require small changes to the programming interface, but the goal of an interface is to minimize the need for such changes. Ultimately, we want to put interfaces between systems to give ourselves flexibility to support new automation, like Tim Toady's self driving car, as easily as we support more manual systems, like Auntie Pat Tern's stick shift.

Real Business Data Returns

Auntie Pat Tern believes you can have your cake and eat it too, "Just use your resources wisely". So I wanted to see how to use my static resources more wisely in my Salesforce org. I have previously shown how to use static resources for test data. Now, I want to use that same data in developer sandboxes created from my org.  Since static resources are brought over like code to developer sandboxes, I can reference them in code I execute when I generate the sandboxes. Using the map datatype helps me keep relationships intact as well:

With this simple trick, I can preload my dev orgs with sample business data that I am using in my unit tests. Like Auntie Pat Tern promised, I can have my cake and eat it too when it comes to using realistic sample data in my tests and development orgs.

Whole Heaps of Fun

Auntie Pat Tern has noticed that questions about heap limits come up periodically. This could be a concern for developers needing to work with files and attachments in Salesforce. Luckily, those native objects aren't subject to typical heap limits. If you stick to blobs you can work with these large data objects. Don't use this knowledge to do bad things, now! Limits help us write code that performs well for our users as well as our fellow tenants in the multi-tenant environment we all know and love.

12:52:35:797 USER_DEBUG [43]|DEBUG|Heap: 57,789,821 / 6,000,000

Of course, if you test your code at limits, you can be sure of just how awesome your code is, which is another great reason to write thorough tests for all your code. You will find that if you are careful of what datatype you use and what you try to do with it the platform rewards you with heaps and heaps of fun.

Eliminate Cargo Cult Programming and Build a Stronger Team

My Auntie Pat Tern is one of those people who refuses to let family members wear cargo pants when we go out to dinner.  She says she doesn't want to participate in the 'cargo cult'.  She inspired me to look at how Cargo Cult Programming has affected our Salesforce org.

One of the best things about programming on the Salesforce platform is the wealth of resources online to help problem solve. It can be one of the worst things about programming on the platform as well if you aren't careful about the resources you choose to follow.  Some code samples use outdated practices, others may be snippets of code taken out of context, which may not behave as expected in a different context.

In one case, the Apex documentation team included System.assert statements in a code snippet.  Our new developer later copied the snippet into a utility class.  The developer found a great resource but didn't fully understand the difference between System.assert and System.debug.  The former builds in a fatal error, appropriate for test code but not for our essential platform automations. Sniffing out this cargo cult programming helped us know what additional training could help our developers.

A more frequently found cargo cult is the factory class to build basic objects. A factory class is still far better than using the very outdated SeeAllData in your test code.  But test data factory classes add unnecessary work for coders. Salesforce offers a newer and better technique with Test.loadData, which I wrote about in July.  It lets us shift the work of creating and maintaining sample data from the developer to the administrator.  

Other cargo cult programming that pops up in our Salesforce org relates to data sharing and security rules.  We teach our developers early on to understand the difference between code that runs only in Triggers vs code that may be run by end-users via UI customizations like pages, flows and actions. And we encourage our administrators to monitor org security in our code during code reviews and test executions.

Hunting out and eliminating cargo cult programming helps us build cohesion among the team that maintains our Salesforce platform and better understanding between administrators and developers who now work more closely together. This helps us spread the effort of org maintenance across a team and make use of clicks and code for our projects. End users benefit and our org is more reliable and easier to maintain as well.