Skip to main content

Self Improving Tasks and Agents

Tasks let Abacus AI Agent do recurring work for you in the background — reports, monitoring, posting, outreach, and more. Self-Improving Tasks go one step further: instead of just repeating the same work, the agent gets measurably better at an objective over time. Each run makes one focused change, an independent evaluation measures the result, and the next run builds on what actually worked.

A self-improving agent is simply an Abacus AI Agent operating inside a Self-Improving Task: it has an objective, a metric, a baseline, and a schedule — and it iterates toward a better number on every run.

You can create and manage both kinds of tasks from the Tasks page in ChatLLM / Abacus AI Agent. If you are new to Tasks, start with the Abacus AI Agent Tasks How-to and the Tasks FAQ.

Tasks vs Self-Improving Tasks​

Regular Tasks are the right tool when you want the same job done reliably on a schedule:

  • A daily news summary in your inbox
  • A weekly sales report from your database
  • Monitoring a website and alerting you on changes
  • Posting scheduled content
  • Updating records in a database

Regular tasks still retain lightweight lessons between runs (what worked, what to avoid), so they get more reliable over time — but they are not driving a number up or down.

Self-Improving Tasks are the right tool when:

  • There is a clear outcome you want to improve (not just work to repeat)
  • That outcome is consistently measurable (a rate, a score, a duration, a count)
  • The agent has access to the measurement data (an API, a database, an eval set, your ratings)
  • You want the agent to make iterative, focused changes and keep what works

Typical objectives: social media engagement, a quality score on a fixed evaluation set, a failure rate, latency, conversion, or human ratings of generated content.

How Self-Improving Tasks Work​

When you create a Self-Improving Task, the agent sets up:

  • Objective — what should get better, in plain language
  • Metric — a single number that captures the objective
  • Direction — whether the metric should increase or decrease
  • Baseline — the current value, measured before any changes
  • Measurement method — a repeatable recipe for computing the metric, fixed up front
  • Schedule — how often improvement runs happen

Then, on every scheduled run, the agent:

  1. Reviews history — what was tried, what was measured, what is carried over
  2. Picks ONE focused improvement — a single deliberate change, not many at once
  3. Applies it — updates content, code, strategy, or configuration
  4. Gets measured — an independent evaluator computes the metric using the fixed measurement recipe; the worker agent cannot grade its own work
  5. Gets graded — improved, unchanged, regressed, or inconclusive
  6. Carries results forward — the next run starts from the measured outcome

If a change regresses the metric, the next run focuses on recovery first. If progress plateaus, the evaluator can pause the task so it stops spending credits — you can resume it anytime from the Tasks page.

Built-In Progress Dashboard​

Every Self-Improving Task gets a progress view on its task page, with no extra setup:

  • The objective, metric, and baseline
  • A trend of the metric across runs
  • Run-by-run history: what changed, what was measured, and the grade
  • The best result so far

You only need a custom dashboard when you want multi-metric or domain-specific views beyond the built-in one.

How to Create a Self-Improving Task​

Describe the outcome in natural language on the Tasks page — no configuration files needed. A good prompt covers:

  • What to improve (the objective)
  • What the agent may change (posts, code, prompts, replies…)
  • How to measure it (the metric and where the data comes from)
  • How often to run
  • Any limits or approvals (e.g. "open a PR, don't merge", "max one post per day")

The agent proposes a plan — objective, metric, measurement method, and a measured baseline — for your approval before the improvement loop starts.

Before letting it run long-term, check that:

  • The agent has real access to the data it needs (connectors authenticated, APIs reachable)
  • The baseline is a real measured value, not a guess
  • The measurement is repeatable (same recipe every run)
  • Only the intended target is being changed
  • The dashboard shows what you care about

Concrete Examples​

1. Grow LinkedIn engagement​

"Post to my LinkedIn once a day about AI trends, and keep improving the engagement rate. Measure engagement as (reactions + comments) per post across the last 7 posts, using my LinkedIn account data."

Metric: engagement rate per post (increase). Needs: LinkedIn connector. Each run adjusts one thing — topic selection, hook style, posting time, format — and keeps what measurably works.

2. Reduce flaky test failures​

"Every night, look at our CI test results and keep reducing the flaky-test failure rate. You may open pull requests with fixes, but never merge them yourself. Measure the failure rate over the last 200 CI runs."

Metric: flaky-failure rate (decrease). Needs: GitHub connector, CI data access. Each run picks the highest-impact flaky test, proposes a fix as a PR, and measures the rate after merges.

3. Improve support answer quality​

"Improve the quality of our support-bot answers on this fixed set of 50 real customer questions. Grade each answer 1–10 against the reference answers, and keep raising the average score. You may change the bot's prompt and knowledge snippets."

Metric: average score on a fixed eval set (increase). Needs: the eval set and reference answers. A fixed evaluation set makes measurement dense and repeatable.

4. Sharpen a product tagline with human feedback​

"Each week, propose one improved tagline for our product and ask me to rate it 1–5. Keep improving the average rating. Base new variants on which past taglines scored well and why."

Metric: your 1–5 rating (increase). Needs: just you — the task sends you a short rating form each run. Human ratings are a perfectly valid metric when no automated measure exists.

5. Cut API response time​

"Keep reducing the p95 response time of our /search endpoint. Run our benchmark script to measure it, and open PRs with one focused optimization at a time. Never merge without approval."

Metric: p95 latency in ms (decrease). Needs: repo access and a benchmark that can run the same way every time.

Choosing a Good Metric​

The metric is the heart of a Self-Improving Task. A good one is:

  • Relevant — moving it actually means the objective is going better
  • Repeatable — the same measurement recipe produces comparable numbers every run
  • Dense — enough data per run to distinguish real change from noise
  • Protected — measured by an independent evaluator with a fixed recipe, so the agent can't redefine success

Prefer a single metric. Use a weighted composite of a few sub-metrics only when the objective genuinely has multiple outcomes that must be balanced.

Credits and Efficiency​

Self-improving runs cost more than regular task runs because every run includes a measurement and grading step. To keep costs down:

  • Use a regular task when you just want recurring work — upgrade it to self-improving later if you decide to optimize an outcome
  • Keep the objective focused (one metric, one target)
  • Pick the lowest frequency that still gives useful signal (daily or weekly beats hourly for most objectives)
  • Prefer APIs and connectors over browser automation when both can get the data
  • Let the task pause on plateau rather than burning runs on a saturated metric

Managing Your Task​

From the Tasks page you can:

  • Open a task to see its runs, progress dashboard, and grades
  • Trigger a manual run
  • Pause, resume, edit, or delete the task
  • Refine the objective or constraints by chatting in the task's conversation

For general task capabilities — schedules, connectors, browser automation, database access, and live dashboards — see the Abacus AI Agent Tasks How-to.