Bull Queue in Node.js: How to Handle Background Jobs with Redis

Leave a Comment / Uncategorized / By

If your Node.js app is starting to feel sluggish because it handles heavy tasks like sending emails, resizing images, generating PDFs or calling slow third-party APIs, you need a background job system. In this Bull queue Node.js tutorial, we will build a real, working setup using Redis, process jobs with retries, and monitor everything with Bull Board.

This is a hands-on guide. By the end, you will have a production-ready pattern you can drop into any Node.js project.

Why Use Bull Queue in Node.js?

Node.js is single-threaded. When you run a CPU-heavy or slow I/O task inside a web request, you block the event loop and your API becomes slow for everyone. Bull solves this by pushing work into a Redis-backed queue that gets processed by separate workers.

  • Reliability: jobs survive restarts because they live in Redis.
  • Retries: failed jobs can retry automatically with backoff.
  • Concurrency: process many jobs in parallel.
  • Scheduling: delayed and repeatable (cron-like) jobs out of the box.
  • Monitoring: a clean dashboard via Bull Board.

Bull vs BullMQ: which one should you pick?

This is the most common question. Here is a quick comparison so you can choose with confidence.

Feature Bull BullMQ
Status Mature, maintenance mode Actively developed
API Callback / Promise based Modern async, class-based
TypeScript Good Excellent
Flows (parent/child jobs) No Yes
Recommended for new projects If stack already uses it Yes

In this tutorial we use the classic bull package because it is still extremely popular and most existing projects use it. The concepts translate almost 1:1 to BullMQ.

redis queue server

Prerequisites

  • Node.js 20 LTS or newer
  • A running Redis instance (local Docker, Upstash, Redis Cloud, etc.)
  • Basic knowledge of Express

Spin up Redis quickly with Docker:

docker run -d --name redis -p 6379:6379 redis:7-alpine

Step 1: Project Setup

mkdir bull-tutorial && cd bull-tutorial
npm init -y
npm install express bull ioredis nodemailer sharp
npm install @bull-board/express @bull-board/api
npm install -D nodemon

Create the basic folder structure:

bull-tutorial/
  src/
    queues/
    workers/
    routes/
    server.js
redis queue server

Step 2: Create the Redis Connection and Queues

Create src/queues/index.js:

const Queue = require('bull');

const redisConfig = {
  redis: {
    host: process.env.REDIS_HOST || '127.0.0.1',
    port: process.env.REDIS_PORT || 6379,
  },
};

const emailQueue = new Queue('email', redisConfig);
const imageQueue = new Queue('image', redisConfig);

module.exports = { emailQueue, imageQueue };

Step 3: Adding Jobs to the Queue

Create src/routes/jobs.js. This Express router accepts requests and pushes jobs to the queue instead of processing them inline.

const express = require('express');
const { emailQueue, imageQueue } = require('../queues');

const router = express.Router();

router.post('/send-email', async (req, res) => {
  const { to, subject, body } = req.body;

  const job = await emailQueue.add(
    'send-welcome',
    { to, subject, body },
    {
      attempts: 5,
      backoff: { type: 'exponential', delay: 3000 },
      removeOnComplete: 100,
      removeOnFail: 500,
    }
  );

  res.json({ jobId: job.id, status: 'queued' });
});

router.post('/process-image', async (req, res) => {
  const { imageUrl, userId } = req.body;

  const job = await imageQueue.add(
    'resize',
    { imageUrl, userId },
    { attempts: 3, priority: 1 }
  );

  res.json({ jobId: job.id, status: 'queued' });
});

module.exports = router;

Notice the important options:

  • attempts: how many times Bull should retry the job before marking it failed.
  • backoff: wait strategy between retries (fixed or exponential).
  • removeOnComplete / removeOnFail: keep Redis clean by trimming old jobs.
  • priority: lower number = higher priority.

Step 4: Creating the Workers

Workers are the processes that actually do the work. Keep them in separate files so you can scale them independently.

Email worker

Create src/workers/email.worker.js:

const { emailQueue } = require('../queues');
const nodemailer = require('nodemailer');

const transporter = nodemailer.createTransport({
  host: process.env.SMTP_HOST,
  port: 587,
  auth: {
    user: process.env.SMTP_USER,
    pass: process.env.SMTP_PASS,
  },
});

emailQueue.process('send-welcome', 5, async (job) => {
  const { to, subject, body } = job.data;

  await job.progress(10);

  const info = await transporter.sendMail({
    from: '[email protected]',
    to,
    subject,
    html: body,
  });

  await job.progress(100);
  return { messageId: info.messageId };
});

emailQueue.on('completed', (job, result) => {
  console.log(`Email job ${job.id} completed`, result);
});

emailQueue.on('failed', (job, err) => {
  console.error(`Email job ${job.id} failed:`, err.message);
});

The number 5 in emailQueue.process('send-welcome', 5, ...) is the concurrency: this worker will process up to 5 emails in parallel.

Image processing worker

Create src/workers/image.worker.js:

const { imageQueue } = require('../queues');
const sharp = require('sharp');
const fs = require('fs/promises');
const path = require('path');

imageQueue.process('resize', 2, async (job) => {
  const { imageUrl, userId } = job.data;

  const response = await fetch(imageUrl);
  const buffer = Buffer.from(await response.arrayBuffer());

  const sizes = [
    { name: 'thumb', width: 150 },
    { name: 'medium', width: 600 },
    { name: 'large', width: 1200 },
  ];

  const outputs = [];
  for (const [index, size] of sizes.entries()) {
    const outPath = path.join('uploads', `${userId}-${size.name}.webp`);
    await sharp(buffer).resize(size.width).webp({ quality: 80 }).toFile(outPath);
    outputs.push(outPath);
    await job.progress(Math.round(((index + 1) / sizes.length) * 100));
  }

  return { outputs };
});
redis queue server

Step 5: Monitoring with Bull Board

Bull Board gives you a clean web UI to inspect waiting, active, completed and failed jobs, and re-run them manually. This is a game changer in production.

Create src/server.js:

const express = require('express');
const { createBullBoard } = require('@bull-board/api');
const { BullAdapter } = require('@bull-board/api/bullAdapter');
const { ExpressAdapter } = require('@bull-board/express');

const { emailQueue, imageQueue } = require('./queues');
require('./workers/email.worker');
require('./workers/image.worker');

const jobsRouter = require('./routes/jobs');

const app = express();
app.use(express.json());

const serverAdapter = new ExpressAdapter();
serverAdapter.setBasePath('/admin/queues');

createBullBoard({
  queues: [new BullAdapter(emailQueue), new BullAdapter(imageQueue)],
  serverAdapter,
});

app.use('/admin/queues', serverAdapter.getRouter());
app.use('/api/jobs', jobsRouter);

app.listen(3000, () => {
  console.log('Server on http://localhost:3000');
  console.log('Dashboard on http://localhost:3000/admin/queues');
});

Run it:

npx nodemon src/server.js

Open http://localhost:3000/admin/queues and you will see your queues live. In a real deployment, protect this route with basic auth or an admin middleware.

Step 6: Scheduled and Repeatable Jobs

Need a job to run every night at 2 AM, or to fire 10 minutes after signup? Bull supports both.

// Delayed: send a reminder 24h after signup
await emailQueue.add(
  'send-welcome',
  { to: '[email protected]', subject: 'Day 1 reminder', body: '...' },
  { delay: 24 * 60 * 60 * 1000 }
);

// Repeatable: nightly cleanup at 02:00
await imageQueue.add(
  'cleanup',
  {},
  { repeat: { cron: '0 2 * * *' } }
);

Production Best Practices

  1. Separate API and workers: deploy workers as their own process or container. Scale them independently.
  2. Always set retries and backoff: external APIs fail, networks blink. Exponential backoff prevents thundering herd.
  3. Use removeOnComplete and removeOnFail: otherwise Redis memory grows forever.
  4. Keep job payloads small: store large data in S3 or a database, pass only the ID in the job.
  5. Make jobs idempotent: a job can be retried, so running it twice must be safe.
  6. Handle graceful shutdown: on SIGTERM, call queue.close() so active jobs finish before the container dies.
  7. Protect Bull Board: never expose it publicly without authentication.
  8. Monitor failed jobs: ship a metric or Slack alert when failed count grows.

Graceful shutdown example

const shutdown = async () => {
  console.log('Shutting down workers...');
  await emailQueue.close();
  await imageQueue.close();
  process.exit(0);
};

process.on('SIGTERM', shutdown);
process.on('SIGINT', shutdown);
redis queue server

Common Pitfalls to Avoid

  • Calling .process() multiple times for the same queue in the same process: this multiplies handlers and causes duplicate execution.
  • Sharing the same Redis DB with cache flushes (FLUSHDB): you will wipe your jobs. Use a dedicated DB index.
  • Forgetting to await job.progress(): not critical, but progress will not update reliably in the dashboard.
  • Putting huge buffers in job data: Redis is not your file storage.

Conclusion

You now have a complete Bull queue Node.js tutorial under your belt: queues, workers, retries, scheduling and a real monitoring dashboard. This pattern works whether you are sending 100 emails a day or processing millions of jobs per week. When you start a brand new project in 2026, consider BullMQ for its modern API and flows, but everything you learned here carries over.

FAQ

Is Bull still maintained in 2026?

Bull is in maintenance mode. The author now focuses on BullMQ. Bull still receives security fixes and is fine for existing projects, but new projects should consider BullMQ.

Can I use Bull without Redis?

No. Bull is built on top of Redis. If you cannot use Redis, look at alternatives like Agenda (MongoDB) or pg-boss (Postgres).

How many workers should I run?

Start with one worker process per queue type and tune concurrency inside it. Scale to more processes only when CPU or throughput requires it. For CPU-heavy work like image processing, one worker per CPU core is a good baseline.

What is the difference between concurrency and multiple workers?

Concurrency processes multiple jobs in parallel inside the same Node.js process (great for I/O bound work). Multiple workers run in separate processes or containers (needed for CPU bound work or horizontal scaling).

How do I retry a failed job manually?

Open Bull Board, go to the Failed tab, and click Retry on any job. Programmatically you can call job.retry().

Can Bull replace a message broker like RabbitMQ or Kafka?

For background jobs inside a Node.js application: yes, easily. For cross-language event streaming at massive scale: no, use Kafka. Bull is purpose-built for job queues, not event sourcing.

Leave a Comment

Your email address will not be published. Required fields are marked *