# Workflows

Background work that survives restarts. A workflow is a function whose steps are saved as they complete: a run that fails is retried with backoff, a run interrupted by a deploy resumes from its last finished step, and every run is listed with its input, output and error under **Workflows** in the PocketBase dashboard.

Workflows run on the OpenWorkflow engine built into Vela's PocketBase, using the [`openworkflow`](https://openworkflow.dev) library directly. Every project created with Vela has them; a project created before they were built in gets them with [`vela enable workflows`](/enable/workflows).

### Syntax

```
$ vela workflows <command>
```

- [`list`](/workflows/list) - List recent runs and their state
- [`run`](/workflows/run) - Start a run from the terminal
- [`cancel`](/workflows/cancel) - Cancel a pending or running run

### How it fits together

`src/lib/server/workflows.ts` holds the client and the worker. It exports `ow`, the OpenWorkflow client your workflows are defined with, `getAdmin()`, a superuser PocketBase client for use inside steps, and `startWorker()`, which the `init` hook in `src/hooks.server.ts` calls once when the server starts.

The worker runs inside the web server, in development and in production alike, so there is no second process to run or deploy. It claims runs from PocketBase and executes them in-process; several servers behind one database share the work safely, since a run is leased to one worker at a time. A project with no workflow files never starts a worker at all.

### Defining a workflow

Workflows live in `src/lib/workflows/`, one file per workflow, and [`vela generate workflow`](/generate/workflow) writes one:

```
$ vela generate workflow send-welcome-email
```

```ts
import { z } from 'zod';
import { ow } from '$lib/server/workflows';

export const sendWelcomeEmail = ow.defineWorkflow(
	{
		name: 'send-welcome-email',
		schema: z.object({ userId: z.string() }),
		retryPolicy: { maximumAttempts: 3 }
	},
	async ({ input, step }) => {
		const user = await step.run({ name: 'load-user' }, async () => {
			const admin = await getAdmin();
			return admin.collection('users').getOne(input.userId);
		});

		await step.run({ name: 'send' }, async () => {
			// ...
		});
	}
);
```

`schema` is what `run()` accepts, checked before the run is queued. Each `step.run` saves its return value, so a retry or a restart carries on after the last step that finished rather than starting over. That makes steps the unit of retry: keep each one safe to repeat. The value a step returns has to be JSON.

A workflow gets one attempt unless `retryPolicy` says otherwise. `step.sleep`, `step.waitForSignal` and child workflows through `step.runWorkflow` are the library's own API and are documented at [openworkflow.dev](https://openworkflow.dev).

### Starting a run

Import the workflow anywhere on the server — a form action, an API route, a hook, another workflow — and call `run()`:

```ts
import { sendWelcomeEmail } from '$lib/workflows/send-welcome-email';

await sendWelcomeEmail.run({ userId: user.id }, { idempotencyKey: user.id });
```

`run()` returns as soon as the run is queued. Its handle has `result()` to wait for the output and `cancel()`; waiting inside a request handler is rarely what you want, since the point is to get the work off the request path.

- `idempotencyKey` - Calling `run()` again with the same key within 24 hours returns the existing run instead of starting another. Key on the thing the run is about — a user id, a Stripe event id — and a retried request or a redelivered webhook cannot do the work twice.
- `availableAt` - A `Date` or a duration such as `'10m'`; the run waits until then.
- `deadlineAt` - A `Date` after which the run is failed rather than retried.

### Recurring workflows

A file that exports a cron expression runs every workflow it defines on that schedule:

```
$ vela generate workflow sync-prices --cron '*/5 * * * *'
```

```ts
export const cron = '*/5 * * * *';
```

The schedule fires once per minute at most, and once across all servers: each firing is keyed on the workflow name and the minute, so however many servers are running, one run starts. A minute that passes while nothing is running is skipped, not caught up. In tests the schedule is off; `tickCron()` from `$lib/server/workflows` starts every recurring workflow on demand.

### Testing

The generator writes a `<name>.server.test.ts` next to each workflow, which [`vela test:server`](/test/server) runs. The test process starts a worker of its own, so a run started in a test completes without going through the dev server:

```ts
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { startWorker, stopWorker } from '$lib/server/workflows';
import { sendWelcomeEmail } from './send-welcome-email';

describe('send-welcome-email', () => {
	beforeAll(() => startWorker());
	afterAll(() => stopWorker());

	it('completes', async (context) => {
		const handle = await sendWelcomeEmail.run({ userId: context.user.id });
		await expect(handle.result({ timeoutMs: 15_000 })).resolves.toBeNull();
	});
});
```

### Settings

Two environment variables tune the worker; neither is required:

```
WORKFLOWS_CONCURRENCY=5
WORKFLOWS_ENABLED=true
```

`WORKFLOWS_CONCURRENCY` caps how many runs one server executes at once. `WORKFLOWS_ENABLED=false` keeps a server from executing runs while still letting it queue them — the switch for moving the work to a dedicated process later.

### In production

The worker lives inside the `vela-web` service, so a [deploy](/deploy) or a restart stops it along with the web server: it finishes the runs it has in flight, and a run it does not finish in time is picked up again after its lease expires, from the last completed step. Runs and their steps are stored in `openworkflow.db` next to the application database, and are part of every [backup](/backup).