Laravel Queue Workers with Redis: Scalable Background Job Processing

Introduction

Modern web applications are expected to respond in milliseconds while simultaneously performing heavy operations such as sending transactional emails, generating PDF invoices, resizing uploaded images, synchronizing data with third-party APIs, and running machine learning inference. Executing these tasks inside a standard HTTP request blocks the PHP-FPM worker, increases latency, and degrades user experience. The moment a request crosses a few hundred milliseconds, conversion rates drop and infrastructure costs rise because you need more application servers to handle the same concurrent load.

The industry-standard solution is to move such work into the background. In the Laravel ecosystem, background work is modeled as queued jobs executed by long-running queue workers. Among the supported queue backends, Redis has become the default choice for teams that need speed, reliability, and operational simplicity. Redis is an in-memory data structure store that exposes atomic list and sorted set operations, making it ideal for building a distributed job broker without the overhead of a separate message queue server.

This article is a complete, production-ready guide to building Laravel queue workers with Redis. We will examine the core concepts, design a robust architecture, walk through a step-by-step implementation, review real-world use cases, study production code examples, compare Redis with alternative drivers, and outline best practices, common mistakes, performance tuning, security, deployment, and debugging. Whether you are launching a new SaaS product or refactoring a legacy monolith, the patterns described here will help you create a system that stays responsive under load and recovers gracefully from failure.

By the end of this guide you will be able to configure Laravel to use Redis as a queue driver, write testable job classes, run supervised workers, scale horizontally across multiple servers, monitor throughput with Laravel Horizon, and avoid the pitfalls that cause silent job loss or memory leaks in long-running PHP processes.

Table of Contents

• Core Concepts
• Architecture Overview
• Step-by-Step Guide
• Real-World Examples
• Production Code Examples
• Comparison Table
• Best Practices
• Common Mistakes
• Performance Tips
• Security Considerations
• Deployment Notes
• Debugging Tips
• FAQ
• Conclusion

Core Concepts

Before writing any code, it is essential to understand the terminology and mechanisms that make Laravel queue workers with Redis function. A job is a PHP class that encapsulates a single unit of work. By implementing the ShouldQueue interface (or using the Queueable trait), you instruct Laravel to push an instance of that class onto a queue instead of executing it synchronously when dispatched.

A queue is a logical channel, identified by a string name such as default, emails, or orders. Each queue is backed by a driver. The Redis driver stores jobs in Redis keys. For a standard queue, Laravel uses a Redis list, for example queues:default. The LPUSH command adds jobs to the head of the list, and BRPOP (blocking right pop) removes jobs from the tail, ensuring first-in-first-out processing.

Delayed or scheduled jobs require a different structure because lists cannot efficiently retrieve items by time. Laravel uses a Redis sorted set (ZSET) named queues:default:delayed. The score of each member is the UNIX timestamp at which the job becomes due. A background timer inside the worker moves due jobs from the sorted set back into the list. Failed jobs are stored either in a database table (failed_jobs) or in another configured store, preserving the payload and exception for later inspection.

A queue worker is a long-running PHP process started by the php artisan queue:work command. The worker boots the Laravel framework once, opens a persistent connection to Redis, and enters a loop: block on BRPOP, deserialize the job payload, instantiate the job class, call its handle() method, and then acknowledge success by removing the job. If the job throws an exception, the worker releases the job back to Redis with a backoff delay or marks it failed after the configured retry limit.

Laravel Horizon is an optional package that provides a dashboard, metrics, and a supervisor specifically for Redis queues. Horizon manages worker pools, auto-balances processes across queues, and records throughput, run times, and failure rates. Supervisor is a generic Linux process manager that restarts workers if they crash; it can be used with or without Horizon.

Other important concepts include job middleware (closures or classes that wrap job execution, such as rate limiters), batch jobs (a group of jobs tracked together), and queue prioritization (processing higher-value queues first by passing an ordered list to the worker).

Architecture Overview

A production-grade Laravel + Redis queue architecture consists of four logical layers: the application layer, the Redis broker, the worker pool, and the failure or monitoring store. In a small deployment, all layers may reside on a single server; in a large system, they are distributed across containers or virtual machines.

The application layer includes web controllers, API endpoints, console commands, and event listeners that dispatch jobs. Dispatching is typically triggered by user actions such as registration, checkout, or file upload. The application serializes the job to JSON (or a configured serializer) and sends an RPUSH or LPUSH command to Redis.

The Redis broker holds the pending jobs in memory. Because Redis is single-threaded for command execution, it can handle tens of thousands of push/pop operations per second, which is more than enough for most Laravel applications. For high availability, Redis can be configured with replicas and Sentinel, or used as a managed service such as Amazon ElastiCache.

The worker pool is a set of queue:work processes. Each worker maintains a persistent TCP connection to Redis and blocks on BRPOP. When a job arrives, one worker receives it, executes the PHP code, and either confirms completion or handles failure. Multiple workers can run on the same machine (using Supervisor numprocs) or across many machines, all pointing to the same Redis instance.

The failure and monitoring store includes the failed_jobs database table and, optionally, Horizon's metrics storage (which also uses Redis). When a job exhausts its retries, Laravel writes a row containing the job class, payload, exception message, and stack trace. Operators review this table via php artisan queue:failed or the Horizon UI and can retry or delete entries.

Data flow in a typical request: (1) User sends HTTP POST to /register. (2) Controller validates input and creates a user. (3) Controller calls SendWelcomeEmail::dispatch($user->id). (4) Laravel serializes the job and pushes it to the queues:emails Redis list. (5) A worker blocked on that queue receives the job, deserializes it, fetches the user from DB, and sends mail. (6) On success, the job is removed; on failure, it is released with delay or moved to failed store.

Step-by-Step Guide

This section provides a clean, repeatable installation on a fresh Laravel 11 project with PHP 8.2 and Redis 7. Commands are written for Ubuntu 22.04 but adapt easily to other distributions.

1. Install System Dependencies

Update packages and install Redis and PHP extensions:

sudo apt updatesudo apt install -y redis-server php8.2-cli php8.2-common php8.2-mbstring php8.2-xml php8.2-curl php8.2-zip php8.2-redisredis-server --version

Ensure Redis is running: sudo systemctl enable --now redis-server and verify with redis-cli ping which should return PONG.

2. Create the Laravel Project

composer create-project laravel/laravel example-appcd example-appphp artisan --version

3. Secure the Redis Instance

Edit /etc/redis/redis.conf to bind to a private interface and set a password:

bind 127.0.0.1requirepass a-very-strong-passwordappendonly yes

Restart Redis: sudo systemctl restart redis-server.

4. Configure Environment Variables

Update .env to use Redis for queues and provide connection details:

QUEUE_CONNECTION=redisREDIS_HOST=127.0.0.1REDIS_PASSWORD=a-very-strong-passwordREDIS_PORT=6379REDIS_CLIENT=phpredis

The REDIS_CLIENT can be phpredis (the C extension) or predis (pure PHP). We recommend phpredis for production due to its speed.

5. Prepare the Failed Jobs Table

Laravel includes a migration for failed jobs. Run it:

php artisan queue:failed-tablephp artisan migrate

6. Create a Job Class

Generate a job that sends a welcome email:

namespace App\Jobs;use App\Mail\WelcomeMail;use Illuminate\Bus\Queueable;use Illuminate\Contracts\Queue\ShouldQueue;use Illuminate\Foundation\Bus\Dispatchable;use Illuminate\Queue\InteractsWithQueue;use Illuminate\Queue\SerializesModels;use Illuminate\Support\Facades\Mail;class SendWelcomeEmail implements ShouldQueue{    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;    public function __construct(public int $userId)    {    }    public function handle(): void    {        $user = \App\Models\User::findOrFail($this->userId);        Mail::to($user->email)->send(new WelcomeMail($user));    }}

7. Dispatch the Job

In a route or controller:

use App\Jobs\SendWelcomeEmail;use Illuminate\Support\Facades\Route;Route::post('/register', function (\Illuminate\Http\Request $request) {    $data = $request->validate([        'email' => 'required|email',        'name' => 'required|string',    ]);    $user = \App\Models\User::create($data);    SendWelcomeEmail::dispatch($user->id)->onQueue('emails');    return response()->json(['status' => 'registered']);});

8. Run a Worker Manually

For development, start a worker in the foreground:

php artisan queue:work redis --queue=emails --tries=3 --sleep=3

In production, you should use Supervisor or Horizon, described later.

9. Install and Configure Supervisor

sudo apt install -y supervisor sudo tee /etc/supervisor/conf.d/laravel-worker.conf <<'EOF'[program:laravel-worker]process_name=%(program_name)s_%(process_num)02dcommand=php /var/www/html/artisan queue:work redis --queue=emails,default --sleep=3 --tries=3 --max-time=3600autostart=trueautorestart=trueuser=www-datanumprocs=4redirect_stderr=truestdout_logfile=/var/log/laravel-worker.logEOFsudo supervisorctl rereadsudo supervisorctl updatesudo supervisorctl start laravel-worker:*

10. Optional: Install Horizon

composer require laravel/horizonphp artisan horizon:installphp artisan migrate

Then run php artisan horizon via Supervisor instead of plain queue:work.

Real-World Examples

Queue workers with Redis power a wide variety of asynchronous workflows. In an e-commerce platform, placing an order might trigger inventory decrement, tax calculation, invoice PDF generation, and a Slack notification. By pushing these to separate queues (orders, invoices, notify), you can assign more workers to orders during flash sales while throttling less critical tasks.

In a social media application, users upload avatars. The web request stores the original image in object storage and dispatches a ResizeImage job. The worker uses GD or Imagick to produce multiple thumbnails, then updates the database with the new paths. The user sees an instant success message while CPU-intensive work happens asynchronously.

For a SaaS billing system, calling Stripe or PayPal APIs should never block a request. A ProcessPayment job handles the API call with retries and rate limiting. If the network flaps, the job is released with exponential backoff, preventing duplicate charges and ensuring eventual consistency.

Another common case is report generation. A manager requests a CSV export of 500,000 rows. The request dispatches a GenerateReport job; the worker streams data to a file, uploads it to S3, and emails a download link when finished. This pattern keeps memory usage flat and timeouts avoided.

Finally, AI inference is increasingly offloaded to queues. A user submits a prompt; the web app puts it on a ai queue; a worker with GPU access calls OpenAI or a local model, stores the result, and notifies the client via WebSockets. Redis queues decouple the slow inference from the request lifecycle.

Production Code Examples

Below are realistic, production-grade snippets. The first shows a robust payment job with retry, backoff, rate limiting, and failure logging.

namespace App\Jobs;use Illuminate\Bus\Queueable;use Illuminate\Contracts\Queue\ShouldQueue;use Illuminate\Queue\SerializesModels;use Illuminate\Queue\InteractsWithQueue;use Illuminate\Foundation\Bus\Dispatchable;use Illuminate\Queue\Middleware\RateLimited;use Throwable;class ProcessPayment implements ShouldQueue{    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;    public $tries = 5;    public $timeout = 60;    public $backoff = [10, 30, 60, 120, 300];    public function __construct(private int $orderId)    {    }    public function handle(): void    {        $order = \App\Models\Order::findOrFail($this->orderId);        app(\App\Services\StripeService::class)->charge($order);    }    public function middleware(): array    {        return [new RateLimited('stripe')];    }    public function retryUntil(): \DateTime    {        return now()->addHours(2);    }    public function failed(Throwable $e): void    {        logger()->error('Payment failed', ['order' => $this->orderId, 'error' => $e->getMessage()]);    }}

The Supervisor configuration for multiple queues and graceful restarts:

[program:laravel-worker]process_name=%(program_name)s_%(process_num)02dcommand=php /var/www/html/artisan queue:work redis --queue=orders,invoices,notify --sleep=3 --tries=5 --max-time=3600autostart=trueautorestart=trueuser=www-datanumprocs=8redirect_stderr=truestdout_logfile=/var/log/laravel-worker.log

Horizon configuration in config/horizon.php:

'environments' => [    'production' => [        'supervisor-1' => [            'connection' => 'redis',            'queue' => ['orders', 'invoices', 'notify'],            'maxProcesses' => 20,            'balance' => 'auto',            'timeout' => 60,        ],    ],],

Docker service definition for a worker in docker-compose.yml:

services:  app:    build: .    command: php-fpm  worker:    build: .    command: php artisan queue:work redis --queue=orders,invoices,notify --tries=5    depends_on:      - redis  redis:    image: redis:7-alpine    command: redis-server --requirepass strongpassword

Batch processing example using Laravel's Bus facade:

use Illuminate\Support\Facades\Bus;use App\Jobs\ProcessRow;$jobs = collect($rows)->map(fn($row) => new ProcessRow($row));Bus::batch($jobs)->onQueue('reports')->dispatch();

Comparison Table

Selecting a queue backend should be a conscious decision. The table below compares Redis with other Laravel-supported drivers across dimensions relevant to production.

Best Practices

• Implement ShouldQueue on I/O-heavy jobs. Any task that touches the network, filesystem, or external service should be queued to keep requests fast.
• Use explicit queue names. Separate workloads by priority and resource profile (e.g., emails, exports, webhooks).
• Configure retries and backoff. Set $tries and $backoff to survive transient failures without hammering dependencies.
• Pass IDs, not models. Store only primitive identifiers in job properties; re-fetch from database in handle() to avoid stale state and large payloads.
• Use Horizon in production. It provides auto-scaling, metrics, and a clear view of stuck jobs.
• Monitor failed jobs daily. A spike in failures often indicates a downstream outage or a code regression.
• Wrap critical logic in transactions. When a job updates multiple records, use database transactions to maintain consistency.
• Rotate Redis credentials. Treat the queue broker as a security boundary; change passwords periodically.
• Set max-time or max-jobs. Periodically restart workers to release memory and reload code after deployments.
• Use job middleware for rate limits. Protect third-party APIs from burst traffic using RateLimited middleware.
• Make jobs idempotent. Design handle() so that processing the same job twice does not cause duplicate side effects.
• Log strategically. Avoid logging full payloads in high-throughput jobs; use structured logs with job IDs.
• Test with queue:fake. In PHPUnit, use Queue::fake() to assert dispatches without running them.
• Version your job payloads. If you change a job's properties, handle old payloads gracefully in handle().

Common Mistakes

• Dispatching inside loops without batching. Calling dispatch() 10,000 times in a loop floods Redis and consumes memory. Use Bus::batch() or chunk.
• Forgetting failed_jobs migration. Without the table, failed jobs vanish, leaving no audit trail.
• Accidental sync driver. A missing QUEUE_CONNECTION=redis in production env causes synchronous execution and timeouts.
• Serializing Eloquent models. Passing a model instance stores its attributes at dispatch time; if the record changes before execution, the job works on stale data.
• No timeout set. A job that hangs (e.g., waiting on an unresponsive API) holds the worker indefinitely. Always set $timeout.
• Using queue:listen in production. queue:listen boots the framework per job; it is slower and should be limited to local debugging.
• Ignoring memory leaks. Long-running workers accumulate memory if you store state in static properties; restart via --max-time.
• Hardcoding queue names. Scattering string literals across code makes refactoring hard; use constants or config.

Performance Tips

• Prefer queue:work over queue:listen. The former keeps the app booted, reducing per-job overhead by orders of magnitude.
• Enable Opcache. Ensure PHP Opcache is enabled on worker containers so compiled bytecode is reused.
• Use phpredis extension. The C extension is significantly faster than the pure PHP Predis client.
• Tune worker count. I/O-bound jobs (email, HTTP) can have many workers per CPU core; CPU-bound jobs should match core count.
• Leverage Horizon balance. The balance option automatically shifts processes to busy queues.
• Use pipelining for bulk dispatch. When dispatching many jobs, wrap in a transaction or use batch to reduce round trips.
• Set appropriate sleep. When a queue is empty, --sleep=3 reduces Redis polling without adding noticeable latency.
• Disable query logging. In jobs, ensure DB::disableQueryLog() if you process thousands of records to save memory.

Security Considerations

Redis is frequently targeted by attackers because misconfigured instances allow unauthenticated access. Never expose Redis to the public internet; bind to 127.0.0.1 or a private network and set requirepass. If Redis is on a separate host, use a security group or firewall rule to restrict port 6379 to application servers only.

Laravel stores job payloads as JSON in Redis. Although JSON is safe against PHP object injection, if you switch to a PHP serializer (not recommended), beware of unserialize vulnerabilities. Always keep the default JSON serializer. Jobs may contain user identifiers or tokens; ensure Redis persistence files are stored on encrypted disks if compliance requires it.

Run worker processes under a dedicated, unprivileged system user (e.g., www-data or queue) with no shell access. Limit the worker's database privileges to only the tables it needs. Finally, treat failed job payloads as sensitive; the failed_jobs table may contain PII, so restrict access and rotate it periodically.

Deployment Notes

For containerized deployments, maintain a single image that serves both web and worker roles. The web container runs PHP-FPM; the worker container uses the same image but overrides the command to php artisan queue:work. In Kubernetes, define a Deployment for workers with a HorizontalPodAutoscaler based on a custom metric (such as queue length exposed by Horizon or a Redis LLEN check).

When using Laravel Forge or Envoyer, configure daemonized workers through the UI or deployment hooks. After each successful deployment, call php artisan queue:restart to signal workers to exit gracefully after finishing current jobs; Supervisor or Horizon will then start fresh workers with the new code.

If you use Terraform or Ansible, codify the Supervisor or systemd unit so that new servers automatically join the worker pool. Ensure that the Redis connection string is injected via secrets management, not hardcoded in version control.

Debugging Tips

• Inspect pending jobs: redis-cli LRANGE queues:default 0 -1 shows raw payloads.
• Check delayed jobs: redis-cli ZRANGE queues:default:delayed 0 -1 WITHSCORES.
• List failed jobs: php artisan queue:failed with IDs and exceptions.
• Retry a job: php artisan queue:retry 123 or queue:retry all.
• Run synchronously in tinker: dispatch_sync(new App\Jobs\SendWelcomeEmail(1)) to reproduce logic without Redis.
• Horizon metrics: visit /horizon to see throughput, slow jobs, and worker status.
• Temporary debug logging: set LOG_LEVEL=debug and add info() calls in handle().

FAQ

What is the difference between queue:work and queue:listen?

queue:work boots Laravel once and keeps the framework in memory, making it ideal for production. queue:listen restarts the framework for every job, which is helpful during development when code changes frequently but is inefficient at scale.

Can I use Redis cluster with Laravel queues?

Yes, but you must configure the redis connection with cluster support. Be aware that some commands used by Laravel (like blocking pop on multiple keys) may have limitations in cluster mode; test thoroughly or use a single primary with replicas behind a proxy.

How do I prioritize queues?

Pass a comma-separated list to --queue: php artisan queue:work redis --queue=high,default,low. Workers will process higher priority queues first, moving to lower priority only when higher ones are empty.

How are delayed jobs stored in Redis?

Laravel uses a sorted set named queues:default:delayed with the execution timestamp as the score. A periodic worker moves due jobs back to the main list. This allows millisecond accuracy without polling the entire dataset.

What happens if Redis goes down?

Dispatched jobs will fail to be stored, throwing an exception in the web request. For resilience, consider a database fallback or use Redis Sentinel with automatic failover so workers can reconnect to a replica promoted to primary.

How do I retry failed jobs?

Use php artisan queue:retry all or specify a job ID from the failed_jobs table. You can also configure automatic retry via the $tries property before marking as failed.

Is Laravel Horizon required for Redis queues?

No. Horizon is an optional monitoring and supervisor tool. You can run plain queue:work with Supervisor. Horizon adds tagging, metrics, auto-scaling, and a web UI, which are highly recommended for production.

How many workers should I run?

It depends on CPU cores and job type. I/O-bound jobs (HTTP, email) can have many workers per core; CPU-bound (image processing) should match core count. Start with 4–8 and tune via metrics.

Can jobs be scheduled?

Yes, using Laravel's task scheduler (php artisan schedule:work) to dispatch jobs at intervals. Alternatively, use the delay() method on a job for one-off future execution.

How do I prevent duplicate job execution?

Design jobs to be idempotent by checking a processed_at flag in the database before performing side effects. You can also use Redis locks inside handle() with Cache::lock() to ensure a single concurrent execution.

Can I dispatch jobs from outside Laravel?

Yes. Any process that can push a properly formatted JSON payload to the Redis list can enqueue a job, but it is easier to expose a small Laravel console command or API endpoint that dispatches the job internally.

Conclusion

Laravel queue workers with Redis provide a battle-tested foundation for asynchronous processing. By following the architecture, code patterns, and operational tips in this guide, you can build systems that remain responsive under load and recover gracefully from failures.

Ready to implement? Start by installing Redis, configuring your .env, and dispatching your first job. For deeper monitoring, install Laravel Horizon and watch your queues thrive. If you found this useful, explore our related articles on Dockerizing Laravel and Redis caching strategies to complete your production stack.