Understand a job

Find the jobs worth a look, see what a job runs, and check whether a change made it cheaper.

Questions you can ask your agent:

  • "Why did nightly_revenue_job cost more this week?"
  • "Which of my jobs are above their usual cost?"
  • "Did nightly_revenue &&& load_orders get cheaper after we changed it on May 1?"

A job id is the orchestrator job, "<dag> &&& <task>", for example "nightly_revenue &&& load_orders". Queries with no job tag are identified by their signature, "SO_SIG||<hash>".

A typical session: jobs_overview when you don't have a job id yet, query_summary_for_job to see what the job runs, then query_detail on its costliest query. After a change, job_run_comparison says whether it helped.

All sample responses use synthetic data and are shortened.

jobs_overview

Lists the jobs worth a look, like the tabs of the web app's Jobs list.

When to use: the user has no job id and asks "what's expensive?", "what changed?" or "what are my jobs?".

Parameters
ParameterTypeRequiredDefaultMeaning
lensstringYesWhich list. See below.
daysintegerNo30The window in days, ending on processing_time. At most 45.
ownerstringNononePart of the owner's name, case-insensitive. Required with by_owner; a filter with the other lenses.
processing_timestring (YYYY-MM-DD)Nolatest loaded dayThe window's last day.
limitintegerNo10Jobs returned. At most 50.

Lenses, each judged on the window's last loaded day:

LensJobs in it
above_usualCost at least the Jobs list's floor that day and 3 times or more the job's usual day, or a new job. Ranked by the cost above usual.
backfills_and_rerunsRuns triggered outside the schedule cost at least the floor that day.
with_findingsThe latest check found something with a savings bound. Ranked by the largest bound.
lost_to_failuresSpark only. The job check found failures with a cost.
held_after_last_taskSpark only. Executors held after the last task, with a cost.
by_ownerJobs whose owner matches owner. Ranked by window cost.

Returns window (start, end, days), matched (jobs in the lens) and jobs, each with job_id, cost, lens_cost, reason, owner, query_user, warehouses and, where the server serves a job health card, next (the call that reads it). An empty lens is an answer: nothing is above usual.

Example

"Which jobs cost more than usual?"

{
  "lens": "above_usual",
  "window": {"start": "2026-04-09", "end": "2026-05-08", "days": 30},
  "matched": 3,
  "jobs": [
    {
      "job_id": "nightly_revenue &&& load_orders",
      "cost": 412.0,
      "lens_cost": 96.5,
      "reason": "4.1x its usual day on 2026-05-08",
      "warehouses": ["ETL_WH"]
    }
  ]
}

query_summary_for_job

Returns a job's queries grouped by statement type, for one day, with 30-day recurrence.

When to use: to summarize what a job does and what each kind of statement costs. Call it once per job; it returns everything.

Parameters
ParameterTypeRequiredDefaultMeaning
job_idstringYes"<dag> &&& <task>", or an untagged query's signature `"SO_SIG
processing_timestringNolatest day the job ran in the last 30The day.

One row per job_tag (the job id and the statement's first word, joined by &&&). Each row has day_count and day_cost for the day, a sample query, and over the last 30 days run_days, triggered_run_days and run_interval_days. Where the orchestrator's query tag allows, rows split runs into scheduled_runs and triggered_runs (by hand, API or script), with scheduled_cost_30d and triggered_cost_30d. statement_versions_30d counts SQL versions; redeploy_cost_30d and redeploy_episodes are the cost of runs that applied changed logic to history.

Zero-cost utility statements fold into one (utility statements) row. A sample query over 6,000 characters keeps its head and tail; query_detail on sample_query_id returns the rest. When nothing is found, the result is {"error": ...} with last_run_day or did_you_mean.

Example

"Summarize nightly_revenue &&& load_orders."

[
  {
    "job_tag": "nightly_revenue &&& load_orders &&& INSERT",
    "day_count": 1,
    "day_cost": 18.42,
    "run_days": 30,
    "scheduled_runs": 1,
    "triggered_runs": 0,
    "statement_versions_30d": 2,
    "sample_query_id": "01b2c3d4-...5555"
  }
]

job_run_comparison

Compares one job's runs before and after a change, on Snowflake or Spark.

When to use: "did my change help?" for a job, with or without a recommendation.

Parameters
ParameterTypeRequiredDefaultMeaning
job_idstringYes"<dag> &&& <task>". May be empty when you pass recommendation_id.
change_datestring (YYYY-MM-DD)NononeBefore is the 14 days before it; after is that day to the latest processed day.
before_startstringNononeExplicit window bound, instead of change_date.
before_endstringNononeExplicit window bound.
after_startstringNononeExplicit window bound.
after_endstringNononeExplicit window bound.
recommendation_idstring (UUID)NononeUse its job, its change date (when none is given: the day it was first marked applied) and the daily saving it claimed.

Give change_date, or the four window bounds, or a recommendation_id.

Returns before and after, each window summed over the job's days with runs (the day's signature cost where covered, else the runs summed). verdict gives the relative change (after over before, less one) of cost per day, cost per run, runs per day and the longest runtime; saved_per_day; and against a recommendation, saved_share_of_claim. too_little_data is set when either window has fewer than 3 days with runs: say the windows are too short to call, not that nothing changed. A lower cost per day with fewer runs per day may be fewer runs, not cheaper ones.

With a recommendation_id, stored_outcome is the measurement the web app's Did this work? tab shows (the same as recommendation_outcome). When present it is the answer; use the windows only as detail.

The answer may carry ask_user (for example "did it ship?" when nobody has answered yet) and end_state (fixed_and_verified, cant_tell_yet or nothing_to_fix). Put the questions to the person; they confirm what shipped on the web app's Did this work? tab.

Example

"Did nightly_revenue &&& load_orders get cheaper after May 1?"

{
  "job_id": "nightly_revenue &&& load_orders",
  "before": {"cost_per_day": 61.0, "cost_per_run": 61.0, "runs_per_day": 1.0},
  "after": {"cost_per_day": 15.9, "cost_per_run": 15.9, "runs_per_day": 1.0},
  "verdict": {
    "cost_per_day_change": -0.74,
    "cost_per_run_change": -0.74,
    "runs_per_day_change": 0.0,
    "too_little_data": false,
    "saved_per_day": 45.1
  }
}

Related


Did this page help you?