Custom Agents

Custom agents use a pull-based model — they poll the controller for work, execute it however they want, and post the result back. Build agents in any language with just an HTTP client. No server needed.

How It Works

Custom agents separate workflow definition from implementation:

• The UI defines the workflow — admins configure task types (name, description, form fields) on the agent card in the dashboard
• The agent code defines the implementation — the agent handles whatever task data it receives

This means you can reconfigure task types without restarting the agent, and non-developers can set up workflows through the dashboard.

Setup

1. Start the agent — it registers with agent_type: "custom" and begins polling
2. Configure task types in the UI — go to Agents page, click the custom agent card, add task types with field definitions, and save
3. Create a job — select "Custom Agent" execution mode, pick your agent, fill in the dynamic form, and trigger

Protocol

All agent endpoints require no API key.

1. Register

POST /api/agents/register
{
    "name": "my-python-agent",
    "tags": ["python", "ml"],
    "hostname": "ml-box",
    "address": "0.0.0.0",
    "port": 0,
    "agent_type": "custom"
}
// Returns: {"agent_id": "uuid", "heartbeat_interval_secs": 10}

2. Discover Task Types (optional)

GET /api/agents/{agent_id}/task-types
// Returns: [{"name": "python", "fields": [...]}, ...]

Useful for logging on startup or validating that your agent handles the configured types.

3. Poll for Work

GET /api/agent-queue/{agent_id}/next
// Returns 204 if no work, or 200 with job JSON:

{
    "queue_id": "uuid",
    "execution_id": "uuid",
    "job_id": "uuid",
    "agent_id": "uuid",
    "task": {
        "type": "custom",
        "agent_task_type": "python",
        "data": {"script": "print('hello')", "args": "--verbose"}
    },
    "callback_url": "http://controller:8080/api/callbacks/execution-result"
}

Your agent switches on task.agent_task_type and reads field values from task.data.

4. Report Result

POST {callback_url}
{
    "execution_id": "...", "job_id": "...", "agent_id": "...",
    "status": "succeeded", "exit_code": 0,
    "stdout": "output", "stderr": "",
    "stdout_truncated": false, "stderr_truncated": false,
    "started_at": "2026-03-25T10:00:00Z",
    "finished_at": "2026-03-25T10:00:05Z"
}

Task Type Definitions

Configured per-agent in the UI. Each definition has a name, optional description, and a fields array:

{
    "name": "python",
    "description": "Run a Python script",
    "fields": [
        {"name": "script", "label": "Script", "field_type": "textarea", "required": true},
        {"name": "args", "label": "Arguments", "field_type": "text", "required": false}
    ]
}

Supported field types: text, textarea, number, select, password

Select fields include an options array with value/label pairs.

Task types can also be managed via the API: GET/PUT /api/agents/{id}/task-types

Queue Behavior

• Jobs are queued in the controller database until the agent polls
• Unclaimed jobs are failed after 5 minutes
• Claimed but unreported jobs are failed after 10 minutes
• The UI shows a "queued" badge (amber) for pending custom agent jobs
• Polling also acts as a heartbeat — no separate call needed

Execution Modes

Mode	Task Types	Targets
Standard	Shell, HTTP, SQL, FTP, Script	Controller / Specific Agent / Any / All
Custom Agent	Defined per agent in UI	Specific custom agent

Python Example

A complete working example is at examples/custom_agent.py:

pip install requests
python3 examples/custom_agent.py

# Configure with:
KRONFORCE_URL=http://controller:8080 \
AGENT_NAME=my-agent \
AGENT_TAGS=python,ml \
python3 examples/custom_agent.py

gRPC Agent Example

A custom agent that calls gRPC services using grpcurl. Supports reflection and proto file discovery.

brew install grpcurl
KRONFORCE_AGENT_KEY=kf_your_key python3 examples/grpc_agent.py

Configure two task types on the agent card:

• grpc_call — call a method (address, service, method, JSON data, optional proto file, plaintext toggle, metadata headers)
• grpc_list — list services or methods via reflection

Scripting

Kronforce supports two script types:

• Rhai Scripts (.rhai) — lightweight scripting language embedded in the Rust binary. Use the script task type.
• Dockerfile Scripts (.dockerfile) — Docker image definitions. Use the docker_build task type to build images and optionally run containers.

Both types are managed in Toolbox → Scripts with syntax highlighting and a reference panel. Select the script type when creating a new script.

Managing Scripts

Use Toolbox → Scripts (with syntax-highlighted editor), the API, or drop .rhai files in the scripts directory.

# Create/update via API
curl -X PUT http://localhost:8080/api/scripts/health-check \
  -H "Authorization: Bearer kf_your_key" \
  -H "Content-Type: application/json" \
  -d '{"code": "let resp = http_get(\"https://api.example.com/health\");\nif resp.status != 200 { fail(\"down\"); }\nprint(\"OK\");"}'

# Or drop a file
echo 'print("hello");' > ./scripts/hello.rhai

Available Functions

Function	Returns	Description
print(msg)	—	Appends to job output
http_get(url)	#{status, body}	HTTP GET request
http_post(url, body)	#{status, body}	HTTP POST request
shell_exec(cmd)	#{exit_code, stdout, stderr}	Run a shell command
env_var(name)	string	Read environment variable
sleep_ms(ms)	—	Sleep for N milliseconds
fail(msg)	—	Mark execution as failed
udp_send(addr, data)	#{sent, error}	Send string via UDP
tcp_send(addr, data)	#{response, error}	Send string via TCP
udp_send_hex(addr, hex)	#{sent, error}	Send raw bytes via UDP
tcp_send_hex(addr, hex)	#{response_hex, response, error}	Send raw bytes via TCP
hex_encode(s)	string	String to hex
hex_decode(hex)	string	Hex to string

Examples

Health check with Slack notification

let resp = http_get("https://api.example.com/health");
if resp.status != 200 {
    http_post("https://hooks.slack.com/services/T00/B00/xxx",
        `{"text": "API is DOWN! Status: ${resp.status}"}`);
    fail("Health check failed");
}
print("API is healthy");

Run command and check output

let result = shell_exec("df -h / | tail -1 | awk '{print $5}'");
print("Disk usage: " + result.stdout);
if parse_int(result.stdout.replace("%", "")) > 90 {
    fail("Disk usage critical");
}

TCP protocol check

let result = tcp_send("192.168.1.10:6379", "PING\r\n");
if result.error != "" {
    fail("Redis unreachable: " + result.error);
}
print("Redis: " + result.response);

Sandboxing

• 1,000,000 operations max (prevents infinite loops)
• 256KB string size max
• Timeout enforced by the job's timeout_secs setting
• No direct file system access (use shell_exec)

Task Types

Type	Execution	Config Fields
shell	Runs sh -c (or sudo -n -u with run_as)	command
http	In-process HTTP request via reqwest	method, url, headers, body, expect_status
sql	Shells out to psql/mysql/sqlite3	driver, connection_string, query
ftp	Uses curl for FTP/FTPS/SFTP	protocol, host, port, username, password, direction, remote_path, local_path
script	Rhai scripting engine	script_name
custom	Dispatched to custom agent	agent_task_type, data (fields defined per agent in UI)
file_push	Base64 decode + write to filesystem	filename, destination, content_base64, permissions, overwrite
kafka	kafka-console-producer via shell	broker, topic, message, key, properties
rabbitmq	amqp-publish via shell	url, exchange, routing_key, message, content_type
mqtt	mosquitto_pub via shell	broker, port, topic, message, qos, username, password
redis	redis-cli PUBLISH via shell	url, channel, message
mcp	Calls a tool on an MCP server via JSON-RPC	server_url, tool, arguments

The first 5 types are built-in and run on the controller or standard agents. The custom type is for custom agents. File Push deploys files to agents. Message queue types (Kafka, RabbitMQ, MQTT, Redis) shell out to CLI tools.

Shell

Runs a command via sh -c. Supports run_as for sudo execution.

{"type": "shell", "command": "echo hello"}

HTTP

Makes an in-process HTTP request. Response body is captured as stdout, status code as exit code.

{"type": "http", "method": "get", "url": "https://api.example.com/health", "expect_status": 200}

SQL

Executes a query via psql, mysql, or sqlite3 CLI tools. Query results are captured as output.

{"type": "sql", "driver": "postgres", "connection_string": "postgresql://user:pass@host/db", "query": "SELECT count(*) FROM orders"}

FTP/SFTP

File transfers via curl. Supports FTP, FTPS, and SFTP protocols.

{"type": "ftp", "protocol": "sftp", "host": "ftp.example.com", "username": "user", "password": "pass", "direction": "upload", "local_path": "/data/report.csv", "remote_path": "/uploads/report.csv"}

Custom

Dispatched to a custom agent. Task types and fields are defined per-agent in the UI. Cannot run locally.

{"type": "custom", "agent_task_type": "python", "data": {"script": "print('hello')", "args": "--verbose"}}

File Push

Upload a file in the job form and deploy it to an agent. Content is base64-encoded in the task JSON (5MB limit).

{"type": "file_push", "filename": "app.conf", "destination": "/opt/app/app.conf", "content_base64": "...", "permissions": "644", "overwrite": true}

Kafka

Publish a message to a Kafka topic via kafka-console-producer.

{"type": "kafka", "broker": "localhost:9092", "topic": "events", "message": "{\"event\":\"test\"}", "key": "user-123"}

RabbitMQ

Publish a message to a RabbitMQ exchange via amqp-publish.

{"type": "rabbitmq", "url": "amqp://guest:guest@localhost:5672", "exchange": "events", "routing_key": "user.created", "message": "{\"user\":\"alice\"}"}

MQTT

Publish a message to an MQTT topic via mosquitto_pub.

{"type": "mqtt", "broker": "localhost", "port": 1883, "topic": "sensors/temp", "message": "22.5", "qos": 1}

Redis

Publish a message to a Redis Pub/Sub channel via redis-cli.

{"type": "redis", "url": "redis://localhost:6379", "channel": "notifications", "message": "{\"type\":\"alert\"}"}

MCP (Model Context Protocol)

Call tools on MCP-compatible servers. Connects to MCP servers via HTTP.

{"type": "mcp", "server_url": "http://localhost:8000/mcp", "tool": "analyze_logs", "arguments": {"path": "/var/log/app.log"}}

Use "Discover Tools" in the job form to browse available tools from a server. Tool results are captured as stdout. The arguments field supports {{VAR_NAME}} variable substitution.

Quick Start with the Test Server

# Install the MCP Python SDK
pip install mcp

# Run the included example server (starts HTTP server)
python3 examples/mcp_test_server.py

Then create an MCP job with:

• Server URL: http://localhost:8000/mcp
• Tool: greet
• Arguments: {"name": "World"}

Available test tools: greet, add, system_info, word_count, reverse.

API Reference

All /api/* endpoints (except health, agent registration, polling, and callbacks) require an API key via the Authorization: Bearer kf_... header.

Jobs

Method	Endpoint	Description
GET	/api/jobs	List all jobs (paginated, filterable by status, searchable)
POST	/api/jobs	Create a job
GET	/api/jobs/{id}	Get job details
PUT	/api/jobs/{id}	Update a job
DELETE	/api/jobs/{id}	Delete a job
POST	/api/jobs/{id}/trigger	Trigger a job now

Executions

Method	Endpoint	Description
GET	/api/jobs/{id}/executions	List executions for a job
GET	/api/executions	List all executions (filterable, searchable)
GET	/api/executions/{id}	Get execution details with output
POST	/api/executions/{id}/cancel	Cancel a running execution

Agents

Method	Endpoint	Auth	Description
GET	/api/agents	Yes	List all agents
GET	/api/agents/{id}	Yes	Get agent details
DELETE	/api/agents/{id}	Yes	Deregister an agent
PUT	/api/agents/{id}/task-types	Yes	Update custom agent task type definitions
GET	/api/agents/{id}/task-types	No	Get task type definitions (for agent discovery)
POST	/api/agents/register	No	Register an agent
GET	/api/agent-queue/{id}/next	No	Poll for work (custom agents)
POST	/api/agents/{id}/heartbeat	No	Agent heartbeat
POST	/api/callbacks/execution-result	No	Report execution result

Events

Method	Endpoint	Description
GET	/api/events	List events (paginated)

Scripts

Method	Endpoint	Description
GET	/api/scripts	List all scripts
GET	/api/scripts/{name}	Get script code
PUT	/api/scripts/{name}	Create or update script
DELETE	/api/scripts/{name}	Delete a script

API Keys

Method	Endpoint	Description
GET	/api/keys	List API keys
POST	/api/keys	Create a key (name, role, optional allowed_groups)
DELETE	/api/keys/{id}	Revoke a key

Group scoping: Set allowed_groups when creating a key to restrict it to specific job groups. Admin keys always see everything. Keys without group restrictions see all groups.

Approval Workflows

Jobs with approval_required: true create a pending_approval execution when triggered instead of running immediately.

Method	Endpoint	Description
POST	/api/executions/{id}/approve	Approve a pending execution (requires write access)

Job Versions

Every job create/update saves a full snapshot. Query the version history for audit trail or rollback reference.

Method	Endpoint	Description
GET	/api/jobs/{id}/versions	List version history (newest first)

Secret Variables

Variables marked as secret have their values masked (••••••••) in API responses and the UI. Secret values are still substituted into task fields at runtime via {{VAR_NAME}}.

Job Templates

Save any job as a reusable template, then create new jobs from it.

• Save — click "Save as Template" on any job's detail page
• Create — click "Template" on the Monitor → Jobs tab to browse and select
• Share — export via GET /api/templates, import via POST /api/templates

Templates capture task config, target, notifications, retry, SLA, and approval settings. Schedule and name are set fresh on each use.

Priority Scheduling

Jobs have a priority field (default 0). Higher priority jobs are scheduled first when multiple are due simultaneously. Set via the job create/update API or the Advanced tab in the job modal.

SLA Deadlines

Set an SLA deadline on a job to track whether it completes on time. Configure in the job modal Advanced tab:

• Deadline — time by which the job must finish (HH:MM, UTC)
• Warning — minutes before the deadline to fire a warning event

When a job with an SLA deadline is still running:

• At deadline - warning_mins: fires sla.warning event (severity: warning)
• At deadline: fires sla.breach event (severity: error)

SLA events trigger notifications (Slack, email, PagerDuty, etc.) and can also trigger other jobs via event-driven scheduling.

OIDC/SSO (Optional)

Set KRONFORCE_OIDC_ISSUER and KRONFORCE_OIDC_CLIENT_ID to enable SSO. The login screen shows a "Sign in with SSO" button alongside API key login.

Method	Endpoint	Description
GET	/api/auth/oidc/config	Check if OIDC is enabled
GET	/api/auth/oidc/login	Redirect to IdP for login
GET	/api/auth/oidc/callback	IdP callback (creates session)
POST	/api/auth/logout	Clear SSO session

Role mapping: Use KRONFORCE_OIDC_ROLE_CLAIM (default: groups) to specify which claim to read. Set KRONFORCE_OIDC_ADMIN_VALUES and KRONFORCE_OIDC_OPERATOR_VALUES to map claim values to roles. Unmatched users get the KRONFORCE_OIDC_DEFAULT_ROLE (default: viewer).

Notifications

Three notification channels, all configured in Settings:

• Email (SMTP) — send to email addresses via any SMTP server
• SMS (Webhook) — send via Twilio or any SMS webhook API
• Webhook (Slack / Teams / PagerDuty) — post to incoming webhook URLs with format-specific payloads

Per-job notification triggers: on failure, on success, on assertion failure. Per-job recipient override or fall back to global recipients.

System alerts: agent went offline notification.

Prometheus Metrics

Scrape GET /metrics (no auth required) for Prometheus-compatible metrics:

• kronforce_executions_total — total executions by status
• kronforce_jobs_total — total number of jobs
• kronforce_agents_total — total registered agents
• kronforce_groups_total — total job groups
• kronforce_jobs_by_task_type — jobs by task type
• kronforce_db_ok, kronforce_db_size_bytes, kronforce_db_wal_size_bytes — database health

Other

Method	Endpoint	Description
GET	/api/health	Health check with DB status (no auth)
GET	/metrics	Prometheus metrics (no auth)
GET	/api/auth/me	Current key/session info
GET	/api/mcp/tools	Discover tools from an MCP server (query: server_url)

Global Variables

Global variables are shared key-value pairs that can be referenced in any task field. Manage them in Toolbox → Variables or via the /api/variables API.

Using Variables in Tasks

Reference variables using {{VAR_NAME}} syntax in any task field — shell commands, HTTP URLs, SQL queries, message bodies, etc.

Task Field	Example
Shell command	curl {{API_HOST}}/health
HTTP URL	{{BASE_URL}}/api/status
Kafka topic	{{ENV}}_events

Variables are resolved controller-side before execution, so they work for both local and remote agent jobs. Shell variables like ${HOME} are unaffected.

Output Extraction Targets

Each extraction rule has a target that controls where the extracted value goes:

• Variable (default) — writes the extracted value to a global variable (set the "Write to var" field). Other jobs can reference it with {{VAR_NAME}}.
• Output — replaces the execution's saved stdout with just the extracted values. Useful for filtering noisy output down to the data you care about.

Patterns without capture groups use the full match. Use (\d+) to extract just a portion, or \d+% to extract the full match.

Updating Variables from Job Output

Set the target to Variable and enter a variable name in the "Write to var" field to automatically update a global variable with the extracted value after each run.

Example pipeline:

1. Job A runs deploy.sh and outputs version=2.4.1
2. An extraction rule captures version with target Variable and write_to_variable DEPLOY_VERSION
3. Job B uses echo "Deployed {{DEPLOY_VERSION}}" in its command

Variable Name Rules

Names must match [A-Za-z0-9_]+ (letters, numbers, underscores only).

Variable Expiration

Variables can have an optional expiration (30, 90, 180, or 365 days). Set this when creating a variable in Toolbox → Variables. Expired variables show a red "expired" badge in the UI. Useful for rotating secrets and temporary configuration that should be reviewed periodically.

Unresolved Variables

If a {{VAR}} reference doesn't match any defined variable, the placeholder is left as-is and a warning is logged. The job still runs.

Connections

Connections are named credential profiles for external systems. Instead of embedding passwords in task definitions, reference a connection by name. Credentials are encrypted at rest using AES-256-GCM.

Supported Types

Type	Config Fields	Used By
PostgreSQL	connection_string	SQL task
MySQL	connection_string	SQL task
SQLite	connection_string (path)	SQL task
FTP / SFTP	host, port, username, password, private_key	FTP task
HTTP	base_url, auth_type, token/username/password/header	HTTP task
Kafka	broker, SASL username/password	Kafka publish/consume
MQTT	broker, port, username, password, client_id	MQTT publish/subscribe
RabbitMQ	url	RabbitMQ publish/consume
Redis	url, password	Redis publish/read
MongoDB	connection_string	Future use
SSH	host, port, username, password, private_key	Future use
SMTP	host, port, username, password	Future use
S3 / MinIO	endpoint, bucket, region, access_key, secret_key	Future use

Using a Connection in a Job

Add a "connection": "my-conn-name" field to any task that supports it. The connection's credentials are merged into the task at execution time. Inline fields take precedence.

{
  "task": {
    "type": "sql",
    "driver": "postgres",
    "query": "SELECT count(*) FROM orders",
    "connection": "prod-db"
  }
}

The connection_string is resolved from the "prod-db" connection at runtime. You only configure the password once.

HTTP Auth Injection

HTTP connections support 4 auth types:

• None — no auth headers injected
• Bearer — adds Authorization: Bearer <token>
• Basic — adds Authorization: Basic <base64>
• Custom Header — adds a named header (e.g., X-API-Key: value)

API

GET    /api/connections              # List all (masked)
GET    /api/connections/{name}       # Get one (masked)
POST   /api/connections              # Create
PUT    /api/connections/{name}       # Update
DELETE /api/connections/{name}       # Delete
POST   /api/connections/{name}/test  # Test connectivity

Sensitive fields (password, token, secret_key, private_key) are masked as ******** in API responses. When updating, send ******** to preserve the existing value.

AI Assistant

When enabled, the Create Job modal shows a "Describe what this job should do" prompt. Type a natural language description and click Generate to have AI fill in all form fields — name, task type, schedule, timeout, notifications, and more.

Setup

Configure AI from Settings → AI Assistant (no restart needed), or via environment variables:

KRONFORCE_AI_API_KEY=sk-ant-...    # Anthropic or OpenAI API key
KRONFORCE_AI_PROVIDER=anthropic    # "anthropic" (default) or "openai"
KRONFORCE_AI_MODEL=claude-sonnet-4-5-20250514  # Optional model override

The AI page is always visible in the sidebar. When no key is configured, it shows setup instructions with disabled controls. No data is sent to the AI provider except the user's description text.

Examples

You type	AI generates
"back up postgres every night at 3am"	Shell task with pg_dump, cron 0 0 3 * * *, timeout 1h, notify on failure
"check if our API is healthy every 5 minutes"	HTTP GET task, cron 0 */5 * * * *, expect 200, Monitoring group
"run ETL extract daily at 6am weekdays only"	Shell task, cron 0 0 6 * * 1-5, ETL group
"clean up temp files older than 7 days every Sunday"	Shell task with find /tmp -mtime +7 -delete, cron 0 0 0 * * 0

How It Works

The AI receives a system prompt describing all Kronforce task types, schedule formats, and configuration options. It returns a JSON job definition that is parsed and used to populate every field in the form. You review and edit before saving — the AI never creates jobs directly.

API

POST /api/ai/generate-job
{"prompt": "back up postgres every night at 3am"}

# Returns:
{"job": {"name": "postgres-backup", "task": {...}, "schedule": {...}, ...}}

Job Groups

Groups let you organize jobs into logical categories (e.g., "ETL", "Monitoring", "Deploys"). Every job belongs to a group — new jobs default to the Default group unless you choose another.

Managing Groups

• Pipelines tab — view all groups in the Stages view, create new groups, rename or delete existing ones
• Job modal — pick a group from the dropdown when creating or editing a job
• Bulk assign — on Monitor → Jobs, select jobs with checkboxes, click "Set Group"

Group Rules

• Names: 1–50 characters, letters, numbers, spaces, hyphens, underscores
• The Default group always exists and cannot be deleted
• Deleting a group moves its jobs to Default
• Renaming a group updates all jobs in that group

API

Method	Endpoint	Description
GET	/api/jobs/groups	List all group names
POST	/api/jobs/groups	Create an empty group ({"name": "ETL"})
PUT	/api/jobs/bulk-group	Assign group to multiple jobs ({"job_ids": [...], "group": "ETL"})
PUT	/api/jobs/rename-group	Rename a group ({"old_name": "ETL", "new_name": "Data Pipeline"})

Filter jobs by group: GET /api/jobs?group=ETL

Execution Retry

Jobs can automatically retry on failure or timeout. Configure retry settings in the Advanced tab of the job create/edit modal.

Settings

Field	Default	Description
Max Retries	0 (disabled)	Number of retry attempts after the initial run
Retry Delay	0 seconds	Initial delay between retries
Backoff Multiplier	1.0 (fixed)	Multiply delay by this factor each attempt (e.g., 2.0 = exponential backoff)

How It Works

• Retries trigger only on Failed or Timed Out status
• No retry on success, cancellation, or assertion failure
• Delay formula: delay_secs × backoff^(attempt - 1), capped at 1 hour
• Each retry creates a new execution linked to the original (retry_of field)
• Execution detail shows "Attempt 2/3" for retry executions

Example

A job with max_retries: 3, retry_delay_secs: 5, retry_backoff: 2.0:

• Attempt 1: runs immediately (fails)
• Attempt 2: waits 5s, retries (fails)
• Attempt 3: waits 10s, retries (fails)
• Attempt 4: waits 20s, retries (succeeds — done)

API

Set retry config on job creation or update:

{"retry_max": 3, "retry_delay_secs": 5, "retry_backoff": 2.0}

Rate Limiting

All API endpoints are rate limited to prevent abuse. Exceeding the limit returns 429 Too Many Requests.

Tiers

Tier	Default Limit	Scope	Endpoints
Public	30/min	Per source IP	Health, dashboard
Authenticated	120/min	Per API key	All authenticated API routes
Agent	600/min	Per API key	Agent register, heartbeat, queue, callbacks

Response Headers

Header	Description
X-RateLimit-Limit	Max requests per minute for this tier
X-RateLimit-Remaining	Requests remaining in current window
Retry-After	Seconds until window resets (on 429 only)

Configuration

Variable	Default
KRONFORCE_RATE_LIMIT_ENABLED	true
KRONFORCE_RATE_LIMIT_PUBLIC	30
KRONFORCE_RATE_LIMIT_AUTHENTICATED	120
KRONFORCE_RATE_LIMIT_AGENT	600

Set any limit to 0 to disable that tier. Set KRONFORCE_RATE_LIMIT_ENABLED=false to disable all rate limiting.

Audit Log

The audit log records all sensitive operations with actor attribution. It is separate from events and has its own retention policy.

What's Recorded

Operation	Resource
key.created, key.revoked	API keys
job.created, job.updated, job.deleted, job.triggered	Jobs
script.saved, script.deleted	Scripts
settings.updated	Settings
variable.created, variable.updated, variable.deleted	Variables
agent.deregistered	Agents
group.renamed	Groups

Each entry records: timestamp, actor (API key name + ID), operation, resource type, resource ID, and details.

API (Admin Only)

Method	Endpoint	Description
GET	/api/audit-log	List audit entries (paginated)

Filters: ?operation=job.created, ?actor=deploy-bot, ?since=2026-03-01T00:00:00Z

Retention

Default: 90 days (separate from events/executions). Configure via Settings: audit_retention_days.

MCP Server

Kronforce exposes an MCP (Model Context Protocol) server that AI assistants and MCP clients can connect to. This lets tools like Claude, Cursor, or custom AI agents discover and manage your jobs through the standard MCP protocol.

Connecting

The MCP server is available at POST /mcp using the Streamable HTTP transport. Connect with your API key:

• URL: http://your-controller:8080/mcp
• Auth: Authorization: Bearer kf_your_key
• Headers: Accept: application/json, text/event-stream

Available Tools

Tool	Description	Min Role
list_jobs	List jobs (filterable by group, status, search)	Viewer
get_job	Get job details by name or ID	Viewer
create_job	Create a new job	Operator
trigger_job	Trigger a job to run now	Operator
list_executions	List recent executions	Viewer
get_execution	Get execution output by ID	Viewer
list_agents	List registered agents	Viewer
list_groups	List job groups	Viewer
list_events	List system events	Viewer
get_system_stats	System overview stats	Viewer

Tools are filtered by the API key's role — viewers see read-only tools, operators can create and trigger jobs.

Configuration

The MCP server is enabled by default. Disable with KRONFORCE_MCP_ENABLED=false.

Cron Expressions

6-field cron with second-level precision: sec min hour dom month dow

Expression	Description
* * * * * *	Every second
0 * * * * *	Every minute
0 0 * * * *	Every hour
0 0 9 * * *	Daily at 9:00 AM
0 0 9 * * 1-5	Weekdays at 9:00 AM
0 */5 * * * *	Every 5 minutes
*/30 * * * * *	Every 30 seconds

Supports: *, ranges (1-5), lists (1,3,5), steps (*/5, 1-30/5).

Day-of-month / day-of-week: follows POSIX OR semantics. When both are set (not *), the job fires if either matches. Example: 0 0 0 15 * 5 fires on the 15th of any month or any Friday.

Schedule Types

Type	JSON	Description
One-shot	{"type": "one_shot", "value": "2026-04-01T00:00:00Z"}	Fires once
Cron	{"type": "cron", "value": "0 * * * * *"}	Recurring schedule
On-demand	{"type": "on_demand"}	Triggered manually only
Event	{"type": "event", "value": {...}}	Fires on system events
Calendar	{"type": "calendar", "value": {"anchor": "last_day", ...}}	Business-day expressions (last day, nth weekday, skip weekends/holidays)
Interval	{"type": "interval", "value": {"interval_secs": 1800}}	Fixed delay from last execution completion

Event Triggers

Jobs can fire reactively when system events occur. Set a job's schedule to "Event" and configure what to listen for.

Available Event Kinds

Kind	When it fires
job.created	A job is created
job.updated	A job is edited
job.deleted	A job is deleted
job.triggered	A job is manually triggered
execution.completed	A job execution finishes (success or failure)
output.matched	An output trigger pattern matched stdout/stderr
agent.registered	An agent registers
agent.offline	An agent heartbeat times out
agent.unpaired	An agent is removed
group.completed	All jobs in a group have succeeded
key.created	An API key is created
key.revoked	An API key is revoked

Trigger Configuration

Field	Description
Event Kind	Which event to listen for. Use wildcards: output.*, job.*, or * for all.
Severity	Optional. Only trigger on events with this severity (error, warning, info, success).
Source Job	Optional. Only trigger when the event came from a job with this name. Matches as a substring.

Example: React to Output Patterns

This is the most powerful pattern — chain jobs based on what another job outputs:

1. Job A ("health-check"): In the Advanced section, add an Output Trigger with pattern ERROR and severity error
2. Job B ("restart-service"): Set schedule to Event, pick output.matched, set severity to error, set Source Job to health-check
3. When Job A runs and its output contains "ERROR", Job B fires automatically

Example: React to Job Failures

1. Job B ("cleanup"): Set schedule to Event, pick execution.completed, set severity to error, set Source Job to etl-pipeline
2. Whenever "etl-pipeline" fails, "cleanup" fires automatically

Extending Kronforce

Kronforce doesn't use a plugin system. Instead, it provides composable primitives that cover most use cases. Here's how common Jenkins plugins map to Kronforce patterns:

Jenkins Plugin Equivalents

Jenkins Plugin	Kronforce Equivalent	How
Git / SCM	Shell task	git clone, git pull in a shell command
Slack Notification	Built-in Slack	Settings → Notifications → Slack webhook URL
Email Extension	Built-in SMTP	Settings → Notifications → SMTP config
PagerDuty	Built-in PagerDuty	Settings → Notifications → routing key
Docker Pipeline	Shell + Dockerfile scripts	Shell task with docker build/run, or Dockerfile script type
HTTP Request	HTTP task	GET/POST/PUT/DELETE with headers, body, auth, assertions
Database plugins	SQL task + Connections	Named connection (Postgres/MySQL/SQLite) + SQL query
SSH / SSH Agent	SSH connection + Shell	Named SSH connection, or shell task with target agent
Credentials Binding	Secret variables + Connections	{{MY_SECRET}} substituted at runtime, encrypted at rest
Pipeline DSL	Dependencies + Events	Job B depends on Job A; event triggers react to completions
Parameterized Build	Job parameters	Define parameters in job config, pass at trigger time
Cron Trigger	Cron schedule	6-field cron expression on any job
Webhook Trigger	Built-in webhooks	Enable webhook on a job, trigger via unique URL
Approval Gate	Built-in approval	Set approval_required: true on any job
Retry / Naginator	Built-in retry	retry_max + retry_delay_secs on any job
Build Timeout	Built-in timeout	timeout_secs on any job
Prometheus Metrics	Built-in /metrics	Scrape endpoint with job/execution counters
LDAP / SAML / OIDC	Built-in OIDC	Okta, Azure AD, Google via OIDC config
Custom Build Step	Custom agent	Write a Python/Go/Node agent for any task type
gRPC / Protobuf	Custom agent	gRPC agent example in docs
ML / GPU Workloads	Custom agent	Run agent on GPU machine, define train-model task type
Kubernetes	Shell task or custom agent	kubectl apply in shell, or K8s-native custom agent
Terraform / Ansible	Shell task	terraform apply, ansible-playbook as shell commands
S3 / Artifact Upload	S3 connection + Shell	Named S3/MinIO connection + aws s3 cp

Custom Agents: The Extension Point

When built-in task types aren't enough, write a custom agent in any language. A custom agent is just an HTTP client that:

1. Registers with the controller (POST /api/agents/register)
2. Long-polls for work (GET /api/agent-queue/{id}/next?wait=30)
3. Executes the task however it wants
4. Posts the result back (POST {callback_url})

Custom agents define their own task types with UI-configurable fields. For example, a Python ML agent might define a train-model task type with fields for dataset URL, model architecture, and hyperparameters. These appear as form fields in the Builder when creating a job targeting that agent.

See the Custom Agents section for the full protocol and examples.

Rhai Scripts: Inline Logic

For lightweight automation that doesn't need an external agent, use Rhai scripts. Rhai is a sandboxed scripting language with built-in HTTP, JSON, math, and string functions. Scripts run on the controller — no agent needed.

Use cases: health checks, API polling, data validation, conditional logic, multi-step HTTP workflows.

See the Scripting section for the API and examples.

Output Rules: Reactive Workflows

Jobs can extract values from their output and trigger downstream actions:

• Extraction — capture regex matches or JSON paths from stdout into variables
• Assertions — fail a job if output doesn't match expected patterns
• Triggers — fire other jobs based on output content
• Forward — POST output to an external URL (webhook relay)

Combined with event triggers and dependencies, these compose into arbitrarily complex workflows without plugins.

MCP Server: AI Tool Integration

Kronforce includes a built-in MCP server that exposes jobs, executions, and agents as tools for AI assistants (Claude, etc.). This lets AI agents trigger jobs, check statuses, and read output programmatically.

API: Build Anything

Every feature in Kronforce is accessible via the REST API. Build custom dashboards, CLI tools, Slack bots, or integrations with any system that speaks HTTP. The API supports pagination, filtering, and real-time streaming (SSE).

crontab -l | kronforce-import-crontab kf_your_key
kronforce-import-crontab kf_your_key --group Monitoring < /etc/cron.d/jobs
kronforce-import-crontab kf_your_key --dry-run < mycrontab

kronforce-import-jenkins kf_your_key Jenkinsfile --pipeline
kronforce-import-jenkins kf_your_key config.xml --group CI-CD
kronforce-import-jenkins kf_your_key ./jenkins-jobs/ --pipeline --dry-run