Bull Queue in Node.js: How to Handle Background Jobs with Redis
If your Node.js app is starting to feel sluggish because it handles heavy tasks like sending emails, resizing images, generating PDFs or calling slow third-party APIs, you need a background job system. In this Bull queue Node.js tutorial, we will build a real, working setup using Redis, process jobs with retries, and monitor everything with Bull Board. This is a hands-on guide. By the end, you will have a production-ready pattern you can drop into any Node.js project. Why Use Bull Queue in Node.js? Node.js is single-threaded. When you run a CPU-heavy or slow I/O task inside a web request, you block the event loop and your API becomes slow for everyone. Bull solves this by pushing work into a Redis-backed queue that gets processed by separate workers. Reliability: jobs survive restarts because they live in Redis. Retries: failed jobs can retry automatically with backoff. Concurrency: process many jobs in parallel. Scheduling: delayed and repeatable (cron-like) jobs out of the box. Monitoring: a clean dashboard via Bull Board. Bull vs BullMQ: which one should you pick? This is the most common question. Here is a quick comparison so you can choose with confidence. Feature Bull BullMQ Status Mature, maintenance mode Actively developed API Callback / Promise based Modern async, class-based TypeScript Good Excellent Flows (parent/child jobs) No Yes Recommended for new projects If stack already uses it Yes In this tutorial we use the classic bull package because it is still extremely popular and most existing projects use it. The concepts translate almost 1:1 to BullMQ. Prerequisites Node.js 20 LTS or newer A running Redis instance (local Docker, Upstash, Redis Cloud, etc.) Basic knowledge of Express Spin up Redis quickly with Docker: docker run -d –name redis -p 6379:6379 redis:7-alpine Step 1: Project Setup mkdir bull-tutorial && cd bull-tutorial npm init -y npm install express bull ioredis nodemailer sharp npm install @bull-board/express @bull-board/api npm install -D nodemon Create the basic folder structure: bull-tutorial/ src/ queues/ workers/ routes/ server.js Step 2: Create the Redis Connection and Queues Create src/queues/index.js: const Queue = require(‘bull’); const redisConfig = { redis: { host: process.env.REDIS_HOST || ‘127.0.0.1’, port: process.env.REDIS_PORT || 6379, }, }; const emailQueue = new Queue(’email’, redisConfig); const imageQueue = new Queue(‘image’, redisConfig); module.exports = { emailQueue, imageQueue }; Step 3: Adding Jobs to the Queue Create src/routes/jobs.js. This Express router accepts requests and pushes jobs to the queue instead of processing them inline. const express = require(‘express’); const { emailQueue, imageQueue } = require(‘../queues’); const router = express.Router(); router.post(‘/send-email’, async (req, res) => { const { to, subject, body } = req.body; const job = await emailQueue.add( ‘send-welcome’, { to, subject, body }, { attempts: 5, backoff: { type: ‘exponential’, delay: 3000 }, removeOnComplete: 100, removeOnFail: 500, } ); res.json({ jobId: job.id, status: ‘queued’ }); }); router.post(‘/process-image’, async (req, res) => { const { imageUrl, userId } = req.body; const job = await imageQueue.add( ‘resize’, { imageUrl, userId }, { attempts: 3, priority: 1 } ); res.json({ jobId: job.id, status: ‘queued’ }); }); module.exports = router; Notice the important options: attempts: how many times Bull should retry the job before marking it failed. backoff: wait strategy between retries (fixed or exponential). removeOnComplete / removeOnFail: keep Redis clean by trimming old jobs. priority: lower number = higher priority. Step 4: Creating the Workers Workers are the processes that actually do the work. Keep them in separate files so you can scale them independently. Email worker Create src/workers/email.worker.js: const { emailQueue } = require(‘../queues’); const nodemailer = require(‘nodemailer’); const transporter = nodemailer.createTransport({ host: process.env.SMTP_HOST, port: 587, auth: { user: process.env.SMTP_USER, pass: process.env.SMTP_PASS, }, }); emailQueue.process(‘send-welcome’, 5, async (job) => { const { to, subject, body } = job.data; await job.progress(10); const info = await transporter.sendMail({ from: ‘[email protected]’, to, subject, html: body, }); await job.progress(100); return { messageId: info.messageId }; }); emailQueue.on(‘completed’, (job, result) => { console.log(`Email job ${job.id} completed`, result); }); emailQueue.on(‘failed’, (job, err) => { console.error(`Email job ${job.id} failed:`, err.message); }); The number 5 in emailQueue.process(‘send-welcome’, 5, …) is the concurrency: this worker will process up to 5 emails in parallel. Image processing worker Create src/workers/image.worker.js: const { imageQueue } = require(‘../queues’); const sharp = require(‘sharp’); const fs = require(‘fs/promises’); const path = require(‘path’); imageQueue.process(‘resize’, 2, async (job) => { const { imageUrl, userId } = job.data; const response = await fetch(imageUrl); const buffer = Buffer.from(await response.arrayBuffer()); const sizes = [ { name: ‘thumb’, width: 150 }, { name: ‘medium’, width: 600 }, { name: ‘large’, width: 1200 }, ]; const outputs = []; for (const [index, size] of sizes.entries()) { const outPath = path.join(‘uploads’, `${userId}-${size.name}.webp`); await sharp(buffer).resize(size.width).webp({ quality: 80 }).toFile(outPath); outputs.push(outPath); await job.progress(Math.round(((index + 1) / sizes.length) * 100)); } return { outputs }; }); Step 5: Monitoring with Bull Board Bull Board gives you a clean web UI to inspect waiting, active, completed and failed jobs, and re-run them manually. This is a game changer in production. Create src/server.js: const express = require(‘express’); const { createBullBoard } = require(‘@bull-board/api’); const { BullAdapter } = require(‘@bull-board/api/bullAdapter’); const { ExpressAdapter } = require(‘@bull-board/express’); const { emailQueue, imageQueue } = require(‘./queues’); require(‘./workers/email.worker’); require(‘./workers/image.worker’); const jobsRouter = require(‘./routes/jobs’); const app = express(); app.use(express.json()); const serverAdapter = new ExpressAdapter(); serverAdapter.setBasePath(‘/admin/queues’); createBullBoard({ queues: [new BullAdapter(emailQueue), new BullAdapter(imageQueue)], serverAdapter, }); app.use(‘/admin/queues’, serverAdapter.getRouter()); app.use(‘/api/jobs’, jobsRouter); app.listen(3000, () => { console.log(‘Server on http://localhost:3000’); console.log(‘Dashboard on http://localhost:3000/admin/queues’); }); Run it: npx nodemon src/server.js Open http://localhost:3000/admin/queues and you will see your queues live. In a real deployment, protect this route with basic auth or an admin middleware. Step 6: Scheduled and Repeatable Jobs Need a job to run every night at 2 AM, or to fire 10 minutes after signup? Bull supports both. // Delayed: send a reminder 24h after signup await emailQueue.add( ‘send-welcome’, { to: ‘[email protected]’, subject: ‘Day 1 reminder’, body: ‘…’ }, { delay: 24 * 60 * 60 * 1000 } ); // Repeatable: nightly cleanup at 02:00 await imageQueue.add( ‘cleanup’, {}, { repeat: { cron: ‘0 2 * * *’ } } ); Production Best Practices Separate
Bull Queue in Node.js: How to Handle Background Jobs with Redis Read More »










