PgFlow.WorkerSupervisor (PgFlow v0.7.0)

Copy Markdown View Source

Supervisor for PgFlow workers.

This module manages worker processes that poll for and execute flow tasks. Sequential starts for the same flow reuse the registered worker on this node.

Workers are GenServer processes that:

  • Compile or verify their flow definition via PgFlow.Worker.Bootstrap on startup
  • Poll pgmq for pending messages
  • Execute step handlers concurrently via Task.Supervisor
  • Report task completion/failure back to pgflow
  • Handle graceful shutdown

Summary

Functions

Returns runtime worker state for a flow module on this node, or nil.

Returns a specification to start this module under a supervisor.

Returns the pid registered for a flow module, if any.

Lists all running workers.

Scales a running worker's runtime limits.

Starts the WorkerSupervisor.

Starts a worker for the given flow.

Stops a worker for the given flow.

Functions

check_worker(flow_module, opts \\ [])

@spec check_worker(module(), keyword()) :: map() | nil

Returns runtime worker state for a flow module on this node, or nil.

Options

  • :timeout - milliseconds (or :infinity) to wait for the worker to reply. Default: 5_000. When it elapses the caller exits with {:timeout, {GenServer, :call, _}}.

Raises ArgumentError on unknown or invalid options.

child_spec(arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

find_worker(flow_module)

@spec find_worker(module()) :: pid() | nil

Returns the pid registered for a flow module, if any.

list_workers()

@spec list_workers() :: [%{pid: pid(), status: :running}]

Lists all running workers.

Returns a list of maps with worker information.

scale_worker(flow_module, opts)

@spec scale_worker(module(), keyword()) ::
  {:ok, map()} | {:error, :not_found | :not_running}

Scales a running worker's runtime limits.

opts takes :max_concurrency and/or :batch_size, plus an optional :timeout (default 5_000 ms, or :infinity) for the call to the worker. Raises ArgumentError on missing or invalid options.

Returns {:error, :not_found} when no worker is registered for the module on this node, or when the worker exits before the scale call completes. When the timeout elapses the caller exits with {:timeout, {GenServer, :call, _}}; the worker may still apply the scale after that.

start_link(config)

@spec start_link(keyword()) :: Supervisor.on_start()

Starts the WorkerSupervisor.

start_worker(flow_module, opts \\ [])

@spec start_worker(module(), keyword()) ::
  {:ok, pid()}
  | {:error, {:conflicting_worker_options, pid()} | :not_started}
  | {:error, term()}

Starts a worker for the given flow.

Options

  • :repo - The Ecto repository (optional, defaults to configured repo)
  • :max_concurrency - Override max concurrent tasks for this worker
  • :batch_size - Override batch size for this worker

Limits resolve from the global config, then the module's worker_options entry, then the supplied options. When a worker is already running, supplied options are compared against its configured (start-time) limits, not limits changed later by scale_worker/2.

Returns {:ok, pid} on success, {:error, {:conflicting_worker_options, pid}} when a worker is already running with different supplied options, {:error, :not_started} when a concurrent supervisor restart started a replacement that exited before it could be returned (retrying is safe), or {:error, reason} on failure. Raises ArgumentError for unknown option keys or invalid values.

A start for a module waits while stop_worker/1 drains that module's worker. When options are supplied and a worker is running, its limits are read with check_worker/1; a worker that does not reply within 5 seconds makes the caller exit with {:timeout, {GenServer, :call, _}}.

stop_worker(flow_module)

@spec stop_worker(module()) :: :ok | {:error, :not_found}

Stops a worker for the given flow.

Gracefully stops the worker, waiting for active tasks to complete, then terminates the supervised child and unregisters it. Normal stops are final under the transient restart policy; crashes and deprecation exits restart.

The stop holds the same per-module lock as start_worker/2, so a concurrent start_worker/2 for this module waits for the drain to finish and then starts a fresh worker instead of returning the one being stopped.