One Batch Apex to rule them all (Part II)

Watch Out!! Bulkified zone!!

In the firsts part of this post, we saw how to isolate the batch apex logic from the operations performed. You can find the full code in the versiononefirst repository.

But today, we are going to extend it further by making our batch apex handler able to perform a list of operations. We’ll also be adding more features and introducing some new concepts as we progress. Let’s go!

The plan

To address the challenge, we’ll change our strategy. We’ll create the BatchableHandler‘s unit tests first, so that we can anticipate and better understand our process needs. Our milestones are:

1. Create this exciting and shining indexed list! Done!
2. Make a unit tests that runs a list of command instances.
3. Make the BatchableHandler able to take a list of commands.
4. Make the BatchableHandler class able to run, one by one, the given operations until there is nothing else to do.

We have a plan! Let’s get our feet wet.

Getting a list of commands

Changing the BatchableHandler to take a list of commands, looks pretty straight forward, but which command instance would you use in the BatchableHandlerTests class to test the process effectively? There are many benefits of writing code based on interfaces. But today we specially will benefit from “receiving” an interface in the constructor and relying on it to perform whatever operation it does. This is known as dependency injection. Thanks to this, we’ll be able to create our own test implementation of a command. This will make the unit test independent from any specific process in our code base. In other words, it will be decoupled!

In our batch apex case, as this is an asynchronous process, we need to make persistent changes we can query and check afterwards to ensure the code is working correctly. I decided to use the Account object, so that I can make some amendments like adding extra text to the existing records’ name. My Batchable test class by now looks like this:

@isTest(SeeAllData=false)
public class BatchableHandlerTests
{
    private class NameAssembler implements Batchable
    {
        public String AccountNameSuffix {get; set;}

        public NameAssembler()
        {
            AccountNameSuffix = '';
        }

        public String getQuery()
        {
            return 'SELECT Id, Name FROM Account LIMIT 1';
        }

        public void execute(List<Account> accounts)
        {
            for (Account acc : accounts)
                acc.Name += ', ' + AccountNameSuffix;

            update accounts;
        }
    }
}

We’ll use AccountNameSuffix property highlighted above to set the aforementioned extra text on the existing accounts’ Name field. So the unit test flow will be: create accounts, ask the BatchableHandler to run the NameAssembler command several times, and finally check that the Name field has changed on all the accounts. E.g.:

@isTest(SeeAllData=false)
public class BatchableHandlerTests
{
    private static List<Account> testAccounts;

    @TestSetup static void createAccount()
    {
        Account testAccount = new Account();
        testAccount.Name = 'Scissorhands, Eduard';
        insert testAccount;
    }

    // Check that BatchHandler successfully runs a couple of commands
    @isTest static void batchableHandler_runsAListOfCommands()
    {
        // Given
        NameAssembler commandA = new NameAssembler();
        commandA.AccountNameSuffix = 'Jack';
        NameAssembler commandB = new NameAssembler();
        commandB.AccountNameSuffix = 'Sparrow';
        List<NameAssembler> commands = new List<NameAssembler>{commandA, commandB};

        // When
        Test.startTest();
        BatchableHandler.runCommands(commands);
        Test.stopTest();

        // Then
        assertAccountNamesContain(commandA.AccountNameSuffix);
        assertAccountNamesContain(commandB.AccountNameSuffix);
    }

Done! You can imagine what the assertAccountNamesContain does, but you can visit the repository later anyway 😉 This is clearly exposing how the BatchableHandler should look and behave. Point 2 done!

To adapt the batch class to this requirement, seems that we have just to amend the constructor to take a list…

public class BatchableHandler implements Database.Batchable<sObject>
{
    private List<Batchable> commands;

    public static Id runCommands(List<Batchable> commands)
    {
        if (commands == null || commands.isEmpty())
            return null; // Or say something relevant

        BatchableHandler batchableHandler = new BatchableHandler(commands);
        return Database.executeBatch(batchableHandler);
    }

    public BatchableHandler(List<Batchable> commands)
    {
        this.commands = commands;
    }
...

And then make the start and execute methods to work with such list. Easy! But wait, we are creating a batch apex class because our operations are very volume demanding. So the BatchHandler class should still be able to run such list with the same performance than it would have when making individual calls per command. Let’s see how.

Chaining commands

This is where the magic happens. Salesforce allows invoking a batch apex class in the finish method. Thanks to this, we can manage one command at a time, like previously, to finally chain a new call to run the next list of operations. This is as simple as the following:

public Database.QueryLocator start(Database.BatchableContext BC)
{
    return Database.getQueryLocator(commands[0].getQuery());
}

public void execute(Database.BatchableContext BC, List<sObject> scope)
{
    // No error handling by now
    commands[0].execute(scope);
}

public void finish(Database.BatchableContext BC)
{
    if (commands.size() > 1)
    {
        commands.remove(0);
        runCommands(commands);
    }
    // else, notify the user that everything is done!
}

Whoa! We have just killed two birds with one stone. We will just process the first command in the list, but in the finish method, we’ll check if there is anything else to do. If so, we have just to pass the list again to the BatchableHandler but removing the first element before.

I can’t wait to see this in action. Let’s run our unit test!

If you still don’t, you’ll love that green tick. Nobody is perfect. I got some failures before getting a successful result, but that’s the best of TDD!! Your work is supported and driven by your unit tests.

Handling background processes

Ok. I still can see some things the BatchableHandler class can’t do. It is not handling errors nor notifying the user when it finishes, so I don’t know if all the chunks have been run.

This is fine for the version one, you know. We’ll extend this further. By the meantime, you can monitor your batch apex jobs execution at the setup page on your org. If you don’t know where, just type apex jobs inside the quick find box and select Apex Jobs.

Jobs executed by unit tests are not shown there, but you can do a test invoking our batch handler in the developer console. Try this for example:

ContactsService.CreateContactsBatchable commandA =
    new ContactsService.CreateContactsBatchable();
ContactsService.CreateContactsBatchable commandB =
    new ContactsService.CreateContactsBatchable();
List<ContactsService.CreateContactsBatchable> commands =
    new List<ContactsService.CreateContactsBatchable>{commandA, commandB};

BatchableHandler.runCommands(commands);

Go back to the apex jobs setup page. There you’ll see something like the following:

It’s alive!! In later posts, we’ll do some improvements to this and filling some gaps the current implementation has. Stay tuned. See you soon!

Conclusions

Command pattern is even more powerful than we saw in the first part. It is designed to deal with any list of operations without knowing what the command does at all. It can even be used to undo operations! But the best of it (at least from my point of view) is how it decouples the operations from the process orchestrator. This is pretty useful, as you have seen, for unit testing. But also to assemble processes that work fine alone without adding dependencies between them. You should always strive for programming interfaces.

Why’s

• Why don’t we make the batch class to implement the stateful interface? That’s a good question, indeed! And there are some reasons why I would keep this non stateful.
  • The batch class is intended to be a generic place to handle volume demanding operations. If we make it stateful, the data stored in memory, to be consumed on next chuncks, would necessarily have to be outside of this class (on the command class). That would make your command method to control states. This is against the idea of making it generic. Your command method should also be generic enough. It should work the same way regardless it is called once or twice (and synchronously or asynchronously). If it needs to remember other executions, maybe it is performing more than one operation. I would suggest to create additional commands.
  • The amount of information on the heap memory, could be out of your control. When working with batch apex, your are supposed to be managing any amount of data. Depending on which kind of data you keep in the heap and for how long, it could make your process to reach the heap size limit. Think that the more possible errors you have to handle, the less maintainable and readable is your code.
• Why do we load data in inside a @TestSetup method? Couldn’t we create it inside the test method? Yes, we could do. It depends on your needs. If all the methods in the class are based in the same set of data, the method annotated with the @TestSetup keyword, saves you execution time. Data is created once, and reseted to its initial state on each method execution. The drawback of using @TestSetup to load your data, is that such information is created in a separate thread. This means that you cannot save the data created in a static variable in your class to use it later. If the data you need to prepare doesn’t need to be in the database, create it within your test.
• Why do we use Test.startTest() and Test.stopTest() to wrap the batch execution? These methods have two uses indeed. One of them is about its meaning. We say that our test starts and finishes at those points. All the code surrounding is just test preparation or result checking. That makes the test method even more meaningful. But there is a second reason to be there. Wrapping the asynchronous process with them, ensures that it has finished before we make our assertions. You’ll need them always when you test batch classes. On the other hand, Test.startTest() resets the Salesforce limits count, so it also helps to check that our tested code doesn’t break by the additional code in the unit test.

Additional resources

A powerful unit testing tool you can use, once you gain more experience with dependency injection, is Paul Hardaker’s apex mocks framework. I encourage you to spend time on it. Although it requires a bit more advanced skills, you must give it a go. You will have no regrets.

Don’t miss the opportunity to check the Jesse Altman’s introduction too, who will be doing a talk about this at Dreamforce! “Speed Up and Simplify Unit Tests with ApexMocks”.

Acknowledgments

Thanks everybody for the warm welcome to the developer bloggers community. Special thanks  to all of you that contributed with your feedback and good advices. Of course, thanks to all of you who made this become socially alive. And thanks to you 🙂

One Batch Apex to rule them all (Part I)

If you have been working with apex some time, you’ll probably know what a Batch Apex is. Even, you probably created more than one batch apex class to support your most demanding processes, right?

But haven’t you being copying and pasting most of the code, to finally amend some bits of it? Don’t they look too similar at the end? Don’t look around, I won’t tell anybody you did it 🙂 It might not be as bad as it seems, because you probably wanted to provide the same user experience and error handling on all your processes. We’ll work on this.

Identifying the problem

Let’s start looking at two very simple batch routine examples.

The first one looks for accounts that don’t have contacts and calls a service class to create a default one:

public class CreateContactsBatch implements Database.Batchable<sObject>
{
    public CreateContactsBatch() {}

    public Database.QueryLocator start(Database.BatchableContext BC)
    {
        return Database.getQueryLocator('SELECT Id, Name FROM Account WHERE Id NOT IN (SELECT AccountId FROM Contact)');
    }

    public void execute(Database.BatchableContext BC, List<sObject> scope)
    {
        ContactsService.createContacts(scope);
    }

    public void finish(Database.BatchableContext BC) {
        // Send a summary
    }
}

The second one, looks for contacts which account is not active. The Active__c field is a ad-hoc custom field:

public class ActivateAccountsBatch implements Database.Batchable<sObject>
{
    public ActivateAccountsBatch() {}

    public Database.QueryLocator start(Database.BatchableContext BC)
    {
        return Database.getQueryLocator('SELECT AccountId, FirstName FROM Contact WHERE Account.Active__c = false');
    }

    public void execute(Database.BatchableContext BC, List<sObject> scope)
    {
        AccountsService.activateAccounts(scope);
    }

    public void finish(Database.BatchableContext BC) {
        // Send a summary
    }
}

Both classes are doing pretty much the same. The things that vary are just the SOQL performed and the method invoked within the execute method. We could spot some other common aspects, but we’ll keep it simply so far.

Ideally it would be great if we could have a unique batch apex class to manage both processes and their behavior. That’s what is this article about. We are going create a single batch class that runs any kind of operation. We are going to apply some good OOP principles, so be prepared!

Applying the command pattern

Our first task is to separate the things that change and leave the common important logic in the batch class. To accomplish this, we’re going to use the command pattern, which aim is to encapsulate method requests.

Our refactoring strategy will be driven by 3 simple steps:

1. Create an abstraction that supplies both operations: SOQL construction and the method operation (the command).
2. Create a Batch class that interacts with the interface.
3. Implement the interface on each Service class that contains the business logic.

First, as we said, we’ll create an abstraction. This will be the Batchable interface as follows:

public interface Batchable
{
    String getQuery();
    void execute();
}

Then, we have just to create a Batch class that delegates the operations on it:

public class BatchableHandler implements Database.Batchable<sObject>
{
    private Batchable command;

    public static Id runCommand(Batchable command)
    {
        if (command == null)
            return null; // Or say something relevant

        BatchableHandler batchableHandler = new BatchableHandler(command);
        return Database.executeBatch(batchableHandler);
    }

    public BatchableHandler(Batchable command)
    {
        this.command = command;
    }

    public Database.QueryLocator start(Database.BatchableContext BC)
    {
        return Database.getQueryLocator(command.getQuery());
    }

    public void execute(Database.BatchableContext BC, List<sObject> scope)
    {
        // No error handling by now
        command.execute(scope);
    }

    public void finish(Database.BatchableContext BC) {
        // We'll add something here to enhance our batch soon
    }
}

Easy, isn’t it? This batch can now deal with any kind of operation which implements the Batchable interface. It ignores what operation is being executed and the data being processed. It is now decoupled from the operations performed! We have additionally simplified the invocation by creating the static runCommand method.

Third (and finally) we have to make the services to implement the interface. Let’s see how to do it for the ContactsService class. You can try doing it yourself for the ActivateAccountsService one.

public with sharing class ContactsService
{
    public class CreateContactsBatchable implements Batchable
    {
        public String getQuery()
        {
            return 'SELECT Id, Name FROM Account WHERE Id NOT IN (SELECT AccountId FROM Contact)';
        }

        public void execute(List<Account> accounts)
        {
            createContacts(accounts);
        }
    }

    public static void createContacts(List<Account> scope)
    {
        // This is the existing logic we already had in the class.
        // Do something here, like create contacts from the given accounts list
    }
}

That’s all! Run it, play with it, make it fail and improve it. I look forward to get feedback from you.

In later posts we’ll enhance this a bit more. For instance, we’ll see how to make the batch class to run several operations using batch chaining. Also, we’ll make the class a bit more handy by adding error handling and notifications.

Stay tuned. See you soon!

Conclussions

Applying this pattern we have immediate benefits from it:

1. Each time we need a new background process, we have just to make the container class to implement the Batchable interface. We don’t need to change the batch class itself. In other words, the batch class is open for extension and closed for modification.
2. Inherited from the point 1, we get a robust framework to make all our processes to be batch capable. It can be tested properly and we won’t need to worry about it again.
3. The batch class can submit any kind of operation without knowing about the underlaying code. We have programmed an interface, not an implementation.

Why’s

• Why did we use the command pattern? Couldn’t we just modify the batch class to be abstract and extend from it whenever we need it? The answer is “yes, of course”. We could use the Template Method pattern instead. However I decided to use the command pattern because composition is always more flexible than inheritance. In next articles we’ll see how we can benefit from this approach by making our batch class to take a list of commands. We’ll be able then to process a queue of requests, which is actually quality of the command pattern.
• Why the batch classes are not using the with/without sharing keywords? This is to provide flexibility. The sharing permissions are inherited from the caller, unless they are specified in the current class. We will be a bit more permissive by letting the caller to decide the security level. As you can see, the ContactsService class uses the with sharing keyword.
• Why is the CreateContactsBatchable an inner class in ContactsService? Could we just make the ContacsService to implement the Batchable interface? Yes, we could. It would asume the class is doing just one thing (create contacts), which is a good practice indeed. However, ContactsService could probably provide more services that interact with contacts in the future. This is up to you.

Welcome to version one first!

Recently, Simon Goodyear (Salesforce MVP), during a Salesforce DUG event, said: “Make the version one first“. That’s a good advice, isn’t it? Developers trend to be too perfectionist, indeed. We never feel happy enough with the code we are witting. We would be improving it forever! That sentence inspired me much and that’s because I’m creating this blog.

There are lot of places around the huge internet world were we, developers, can find tutorials, introductions, experiences, experiments, etc, about the Salesforce technology and development in general. Then, why another one?

Because, the three w’s have the information, but finding what you are looking for, and in the desired degree of depth, takes long. This is why I’ll be creating articles, in a level based way. As you have just guessed, there will be a place for the version one, being that the one with less knowledge requirements and followed by a more elaborated ones. Hopefully, some of them will fulfill your needs.

On the other hand, I have been seduced by the irresistible need of express my ideas and findings to the vast world of dev community. Hey mom! Look what I can do! So I’ll do a bit of therapy here 🙂

There will be a place for the other bloggers I admire much. I’ll gather here some of my very favorite blogs in my catablog.

Hope you find useful stuff here. Welcome, and enjoy!