Job Batches in Laravel

By /
Job Batches in Laravel

When is it helpful to utilize job batches in Laravel?

We’ve touched on queues in a previous article. Queues are a useful tool in Laravel, allowing you to run complicated or time consuming tasks in the background. There will be times when you need to run dozens, or even hundreds of instances of a job — for instance, creating individual reports for a larger number of clients, or importing a folder of .csv files.

This could be accomplished with running a single job with a loop to run the required logic. The problem with this approach is scalability. It will probably work fine if you have small number of reports to create or files to import, but at certain point, execution time and memory usage for the job will exceed the limits you’ve defined in your PHP and server configuration.

While you could increase those limits, a more efficient approach would be to run each report as a separate job — which has a relatively small execution time and memory usage — then create a .zip file when all jobs complete. This can be accomplished using Job Batching.

What is Job Batching?

Job Batching is a feature in Laravel. As the name implies, it allows you to execute a batch of jobs. Data about the batch is stored in a table called job_batches; this contains useful information about the batch, such as the total number of jobs, the number of failed jobs, the number of jobs still in progress, etc. This table is created using the following commands:

php artisan queue:batches-table
php artisan migrate

In addition, you can define a number of callback functions that run when certain events occur. These include:

  • Progress – runs when a single job is successfully completed
  • Then – runs when all jobs are successfully completed
  • Catch – runs the first time a job failure is detected
  • Finally – runs when all jobs have been executed (successfully or not)

Creating a Job Batch

In order to demonstrate what you can accomplish with batches we’ll use the example of creating a monthly Invoice as a PDF for each of the clients in a hypothetical billing system. For our purposes, the invoices will be saved to a .zip folder to be reviewed before they are sent. The workflow will look something like this:

  1. Find all clients that need an invoice.
  2. Loop through each of the clients, generate the invoice, and save the PDF on the server.
  3. Zip the invoices into a single file.
  4. Email the user who requested the invoices with a link to the .zip file.
  5. If any of the PDFs fail, cancel and inform the user that there was an issue.

We’ll keep things simple and assume there is an invoice model that has methods to find the list of clients and generate the invoice. We’ll also keep the error handling pretty minimal and just send an email if any of the invoices fail; we’ll touch on some improvements we can make later.

Creating the Invoice Job

Starting off, we’ll create the job for generating a single PDF invoice. We’ll pass the invoice ID, and, in the handler, find the appropriate invoice and call the method to generate and save the invoice PDF. In order to make it a batchable job, we will need to add the Illuminate\Bus\Batchable trait to the job class. This trait provides access to a batch method which will allow us to retrieve the batch the job belongs to.

We’ll start by creating the a job called CreateInvoice using an artisan command.

php artisan make:job CreateInvoice

After adding the Illuminate\Bus\Batchable trait and the code to generate the invoice, the job class looks like this:

<?php

namespace App\Jobs;

use Illuminate\Bus\Batchable;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use App\Models\Invoice;

class CreateInvoice implements ShouldQueue
{
    use Batchable, Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    private $invoice_id;

    /**
     * Create a new job instance.
     */
    public function __construct($invoice_id)
    {
        $this->invoice_id = $invoice_id;
    }

    /**
     * Execute the job.
     */
    public function handle(): void
    {
        if ($this->batch()->cancelled()) {
            // Determine if the batch has been cancelled...

            return;
        }
        
        $invoice = Invoice::find($this->invoice_id);
        $invoice->createPDF();
    }
}

The handler now consists of 3 main parts:

  1. An if statement that detects if the batch has been cancelled — this most often occurs if a job fails. This prevents any further invoices from being generated in case any of them fail.
  2. Creating an invoice object using the ID passed into the job
  3. Calling the createPDF method in the invoice model to create a PDF version of the invoice and save it to disk.

Running the Batch

In order to run our new batch job, we will need to call the batch method of the Illuminate\Bus\Batch façade. We’ll start with a basic call, just creating a handful of invoices as an example:

use App\Jobs\CreateInvoice;
use Illuminate\Bus\Batch;
use Illuminate\Support\Facades\Bus;
use Throwable;

$batch = Bus::batch([
    new CreateInvoice(1),
    new CreateInvoice(2),
    new CreateInvoice(3),
    new CreateInvoice(4),
])->then(function (Batch $batch) {
    // All jobs completed successfully...
})->catch(function (Batch $batch, Throwable $e) {
    // First batch job failure detected...
})->dispatch();

We’ve called the batch function, passing in four CreateInvoice jobs. We’ve also defined the then callback, where we’ll zip up the files and send a link to the file, and the catch callback, where we’ll send an error message to the user in case one of the jobs fails.

Of course, we’re going to want to create more than four invoices. We’ll abstract things again, and assume we have a static method in the invoice model, called getInvoicesByMonth, that returns a list of invoices for a specific month, as well as a method, called createZipFile, that will zip the invoices up and return a link to the resulting file. Now we’ll create a new batchable job called LoadInvoiceBatch:

<?php

namespace App\Jobs;

use App\Models\Invoice;
use Illuminate\Bus\Batchable;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class LoadInvoiceBatch implements ShouldQueue
{
    use Batchable, Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    private $month;

    /**
     * Create a new job instance.
     */
    public function __construct($month)
    {
        $this->$month = $month;
    }

    /**
     * Execute the job.
     */
    public function handle(): void
    {
        if ($this->batch()->cancelled()) {
            return;
        }

        $invoice_list = Invoice::getInvoicesByMonth($this->month);
        foreach ($invoice_list as $invoice) {
            $this->batch()->add(new CreateInvoice($invoice->id));
        }
    }
}

With this in place, we can run our new job and define our callbacks:

use App\Jobs\LoadInvoiceBatch;
use App\Models\Invoice;
use App\Notifications\InvoiceError;
use App\Notifications\InvoicesCompleted;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Auth;
use Illuminate\Support\Facades\Bus;

$batch = Bus::batch([
    new LoadInvoiceBatch($month)
])->then(function() {
    $link = Invoice::createZipFile();
    Auth::user()->notify(new InvoicesCompleted($link));
})->catch(function() {
    Auth::user()->notify(new InvoiceError());
})->dispatch();

As part of creating the batch, we’ve stored it in a variable called $batch; this has a number of useful properties and methods that give us information about the batch, including the batch ID, the total number of jobs in the batch, the number of pending jobs, the number of failed jobs, the percentage of jobs complete, and more. There is also a method to cancel the job manually (if you need to). This example was fairly simple; there is more you can do with batches, such as attempting to retry any failed jobs in the batch. You can find more information in the Laravel Documentation.