bullmq

BullMQ

基于 Redis 的 Node.js 队列系统。包含四个核心类：Queue、Worker、QueueEvents、FlowProducer。

目录

• 安装
• 快速开始
• 连接
• 队列
• Worker
• TypeScript 泛型
• 事件
• 任务生命周期状态
• 高级主题

安装

yarn add bullmq — 需要 Redis 5.0+，并将 maxmemory-policy=noeviction 设为必需。

<quick_reference>

快速开始

import { Queue, Worker, QueueEvents } from "bullmq";

// --- Producer ---
const queue = new Queue("my-queue", {
  connection: { host: "localhost", port: 6379 },
});

await queue.add("job-name", { foo: "bar" });

// --- Consumer ---
const worker = new Worker(
  "my-queue",
  async (job) => {
    // process job
    await job.updateProgress(50);
    return { result: "done" };
  },
  { connection: { host: "localhost", port: 6379 } },
);

worker.on("completed", (job, returnvalue) => {
  console.log(`${job.id} completed with`, returnvalue);
});

worker.on("failed", (job, err) => {
  console.error(`${job.id} failed with`, err.message);
});

// IMPORTANT: always attach an error handler
worker.on("error", (err) => {
  console.error(err);
});

// --- Global event listener (all workers) ---
const queueEvents = new QueueEvents("my-queue", {
  connection: { host: "localhost", port: 6379 },
});

queueEvents.on("completed", ({ jobId, returnvalue }) => {
  console.log(`Job ${jobId} completed`);
});

queueEvents.on("failed", ({ jobId, failedReason }) => {
  console.error(`Job ${jobId} failed: ${failedReason}`);
});

任务生命周期状态

add() → wait / prioritized / delayed
         ↓
       active → completed
         ↓
       failed → (retry) → wait/delayed

使用 FlowProducer 时，任务也可以处于 waiting-children 状态，直到所有子任务完成。

</quick_reference>

连接

BullMQ 内部使用 ioredis。传入 connection 选项或现有的 ioredis 实例。

import { Queue, Worker } from "bullmq";
import { Redis } from "ioredis";

// Option 1: connection config (new connection per instance)
const queue = new Queue("q", {
  connection: { host: "redis.example.com", port: 6379 },
});

// Option 2: reuse ioredis instance (Queue and multiple Queues can share)
const connection = new Redis();
const q1 = new Queue("q1", { connection });
const q2 = new Queue("q2", { connection });

// Option 3: reuse for Workers (BullMQ internally duplicates for blocking)
const workerConn = new Redis({ maxRetriesPerRequest: null });
const w1 = new Worker("q1", async (job) => {}, { connection: workerConn });

重要规则：

• Worker 必须在 ioredis 实例上设置 maxRetriesPerRequest: null。BullMQ 会强制执行此要求；如果未设置，会发出警告或抛出异常。
• 不要使用 ioredis 的 keyPrefix 选项；请改用 BullMQ 的 prefix 选项。
• QueueEvents 不能共享连接（会使用阻塞式 Redis 命令）。
• Redis 必须设置 maxmemory-policy=noeviction。

队列

const queue = new Queue("paint", { connection });

// Add a job
await queue.add("job-name", { color: "red" });

// Add with options
await queue.add(
  "job-name",
  { color: "blue" },
  {
    delay: 5000, // wait 5s before processing
    priority: 1, // lower = higher priority (0 is highest, max 2^21)
    attempts: 3, // retry up to 3 times
    backoff: { type: "exponential", delay: 1000 },
    removeOnComplete: true, // or { count: 100 } to keep last 100
    removeOnFail: 1000, // keep last 1000 failed jobs
  },
);

// Add bulk
await queue.addBulk([
  { name: "job1", data: { x: 1 } },
  { name: "job2", data: { x: 2 }, opts: { priority: 1 } },
]);

// Queue operations
await queue.pause();
await queue.resume();
await queue.obliterate({ force: true }); // remove all data
await queue.close();

Worker

const worker = new Worker<MyData, MyReturn>(
  "paint",
  async (job) => {
    await job.updateProgress(42);
    return { cost: 100 };
  },
  {
    connection,
    concurrency: 5, // process 5 jobs concurrently
    autorun: false, // don't start immediately
  },
);

worker.run(); // start when ready

// Update concurrency at runtime
worker.concurrency = 10;

处理器接收 3 个参数： (job, token?, signal?) — signal 是用于支持取消操作的 AbortSignal。

TypeScript 泛型

interface JobData {
  color: string;
}
interface JobReturn {
  cost: number;
}

const queue = new Queue<JobData, JobReturn>("paint");
const worker = new Worker<JobData, JobReturn>("paint", async (job) => {
  // job.data is typed as JobData
  return { cost: 100 }; // must match JobReturn
});

事件

Worker 事件（仅对该 Worker 实例生效）：

事件	回调签名
completed	(job, returnvalue)
failed	(job | undefined, error, prev)
progress	(job, progress: number | object)
drained	() — 队列为空
error	(error) — 必须附加此处理器

QueueEvents（全局事件，适用于所有 Worker，使用 Redis Streams）：

事件	回调签名
completed	({ jobId, returnvalue })
failed	({ jobId, failedReason })
progress	({ jobId, data })
waiting	({ jobId })
active	({ jobId, prev })
delayed	({ jobId, delay })
deduplicated	({ jobId, deduplicationId, deduplicatedJobId })

事件流会自动修剪（约 10,000 个事件）。通过 streams.events.maxLen 配置。

高级主题

• 任务类型和选项（延迟、优先级、去重、可重复）：参阅 references/job-types-and-options.md
• 流程和调度器（FlowProducer、父子任务、任务调度器、cron）：参阅 references/flows-and-schedulers.md
• 模式（分步任务、幂等、节流、手动限流）：参阅 references/patterns.md
• 生产环境（关闭、Redis 配置、重试、退避、监控）：参阅 references/production.md