Documentation Docs version: dev (Pre-release: Gitoodo has not been released yet)

For developers · App internals

Logging

On this page
  1. Location
  2. Levels
  3. Rotation, budget and retention
  4. Service events
  5. Runs and steps
  6. Reading runs
  7. Questions
  8. Worker transcripts
  9. Event types
  10. Audit events
  11. Throttling
  12. Debugging

Gitoodo writes a structured log to help explain what happened on a user's machine (for example a token check that failed, a worker that did not start, or a sync that keeps failing). The app and every background service (classifier, dispatcher with its worker supervision, satellite) use the same module (crates/core/src/logging) and schema. The log never contains tokens, Authorization headers, response bodies, or URL paths and queries; servers are logged as host[:port] only.

Location#

One folder per user, shared by the app and the services:

SystemFolder
macOS~/Library/Logs/de.gitoodo.app/
Windows%LOCALAPPDATA%\de.gitoodo.app\logs\
Linux$XDG_STATE_HOME/gitoodo/logs/ (~/.local/state/...)

GITOODO_LOG_DIR overrides the folder; with GITOODO_STATE_DIR set (tests, isolated runs) it is <state>/logs. The test-access build keeps its own folder under its identifier.

FileWriter
gitoodo.logthe app
classifier.jsonlgitoodo classifier (also when the app starts it)
dispatcher.jsonlgitoodo dispatcher, including the workers it supervises
satellite.jsonlgitoodo satellite (not the --stdio bridge)
<service>.stderr.logstderr of a service (bootstrap errors, a crash before logging started)
runs/<run_id>/…terminal transcript of one worker run (see Worker transcripts)

Each process owns its file through a sidecar lock (<file>.lock). A second process of the same kind (for example a classifier service waiting for the app's classifier) writes <name>-<pid>.<ext> instead, so two processes never rotate one file.

Each line is one JSON object (schema v 2):

{
  "v": 2,
  "ts": "2026-11-16T11:00:11.175Z",
  "seq": 12,
  "level": "warn",
  "proc": "dispatcher",
  "pid": 4711,
  "component": "dispatch",
  "event": "dispatch.host_wait",
  "cid": null,
  "run_id": null,
  "issue": "owner/repo#42",
  "span": null,
  "audit": false,
  "msg": "waiting for a host; no matching host can take it",
  "ctx": { "host": "mac-mini", "reason": "offline" }
}
  • ts: UTC, RFC 3339 with milliseconds. seq: line number within the process, so gaps show dropped lines.
  • proc: app, classifier, dispatcher or satellite; pid the process ID.
  • event: stable name; service events are dotted (<area>.<what>), app events keep their earlier names. msg is human text, never parsed.
  • cid is a correlation ID: every line of one sync run (or one command) shares it. A cid field on an event (a classifier decision the dispatcher refers to) is lifted out of ctx.
  • run_id, issue and span are top-level for every line (null when unknown): a field of that name on an event is lifted out of ctx. run_id and span are reserved for timed worker runs (see Runs and steps).
  • ctx holds the remaining typed fields (at most 32), each redacted.

Levels#

LevelMeaning
errorUnrecoverable failure the user relies on (token check failed, keychain write failed)
warnFailed, but retried or carried on (sync failed, integration error, keychain read failed)
infoState changes (sync finished, settings changed, audit events)
debugDetail (sync started, every keychain access)

Dependencies log only warn and error.

Rotation, budget and retention#

  • Per file: 5 MiB, then the file becomes <file>.1 (older generations shift, beyond .3 they are deleted): at most 4 files and 20 MiB per process. A line that would not fit after a failed rotation is dropped rather than written past the cap (on Windows a file held open by a viewer cannot be renamed; Gitoodo then copies and empties it, and if that fails too it drops lines and retries after 30 s). The next written line reports the count as dropped_line.
  • Whole folder: a disk budget, default 75 MiB (setting log_budget_mb, 20–500), covering every log and stderr file in the folder and one level of subfolders. Files older than 14 days (setting log_max_age_days, 1–90) are deleted. When the folder is over budget, the oldest files that no running process writes are deleted first. *.stderr.log files are emptied above 1 MiB.
  • When: at the start of every process and every 6 h after (log.cleanup names what was removed). Files of a running process are never deleted; their size is bounded by the per-file cap.

The status window shows the folder's size against the budget.

Service events#

AreaEvents
every serviceservice.started (version), service.stopped, service.fatal (error), panic (location), log.cleanup
classifier (polls)classifier.cycle (duration_ms, accounts, repos, errors, forge_requests, forge_failures, forge_rate_limited, forge_retry_wait_ms, forge_p50_ms, forge_p95_ms, forge_max_ms) after every poll
dispatcher (forge)dispatch.forge_summary (the same forge_* fields) every 5 min when it made requests
dispatcher (control)control.received (cmd, action, issue), control.applied (result, duration_ms, error): park, stop, restart, attach and worker requests at info, queries at debug; never texts or capabilities
workersworker.transcript (path), worker.transcript_failed, worker.transcripts_recovered (count)
classifierclassifier.started, .lock_wait, .lock_taken, .handover, .parent_gone, .poll_error, .account_error, .reconciled
classifier (runs)classifier.ready (cid, issue, previous): an issue became ready
dispatcherdispatch.started, .worker_started, .worker_exit, .start_failed, .host_wait (issue, reason, since, detail), .wait (reason, since), .hung, .merged, .merge_failed, .lease_*, .cleanup, .shutdown
runsrun.start, run.end, span.start, span.end, satellite.clock (see below)
automations (in dispatcher)automation.started, .postponed, .rate_limited, .gave_up, .config_error
satellitesatellite.listening, .worker_started, .worker_exit (run_id, code, duration_ms), .spawn_failed, .kill, .delivery_failed, .subscription_lost, .shutdown

Fields used across events: issue (issue key), host, repo, error (redacted text), reason.

Runs and steps#

One attempt of the dispatcher at an issue is a run with its own ID, r- and 10 lowercase Crockford base32 characters (r-7k2m9q4xzt). Every line of the attempt carries it in run_id, whichever process writes it:

  • The classifier logs classifier.ready with a fresh cid when an issue becomes ready and keeps that decision (cid, since when) in the read model while the issue stays ready.
  • The dispatcher creates the run ID when it picks the issue up and logs run.start with that cid, host, worker and agent, so the run is linked to the decision.
  • The ID is stored in the worktree lease, sent to a satellite in the spawn request and given to the worker as GITOODO_RUN_ID. Satellite lines of the worker (its worktree, spawn and satellite.worker_exit) carry it, and so does the main host's delivery (pr.open) of a satellite branch. The pull request itself does not carry it.
  • run.end closes the run with duration_ms and result: merged, stalled, needs_human, parked, paused, user_stop, start_failed, lease_refused, host_gone or skipped.

Each step is a span. A step that runs within one call is logged as span.start and span.end; a step that already happened (queue.wait, host.select, merge) as one span.end. span.end always has duration_ms and result (ok, error, cancelled, timeout); otherwise also error_kind and, where useful, error (redacted). A run that ends with steps still open ends them with the run's outcome.

StepFrom → toError kinds
queue.waitthe classifier's ready decision → the dispatcher's pickup (since)–
host.selectchoosing the host among the ready issues of the repo–
clone.ensurefinding the working copy, cloning it when missing, fetching the base (local hosts)clone_failed
worktree.createcutting the worktree (on the dispatcher or the satellite)worktree_failed
worker.spawnstarting the worker (for a satellite: the whole spawn request)spawn_failed, satellite_refused, …_unreachable
worker.runworker start → its exit (restarts reopen it)exit_code, hung, the run's result
pr.opengitoodo worker pr: push and pull requestdelivery_failed
checks.waitpull request found → checks decidedchecks_failed, the run's result
mergethe merge call (or enabling auto-merge)merge_failed
merge.waitauto-merge or a human merge → merged–
cleanupremoving the worktree and the branch after the merge (local hosts)cleanup_failed

Waits are logged once per change, not on every pass: dispatch.wait (the dispatcher starts no worker: paused, no_dispatching_repo, classifier_unreachable) and dispatch.host_wait (an issue's os:/host: labels match no free host, with the concrete reason of #750: unknown_host, os_mismatch, offline, agent_missing, usage_gate, low_memory, at_capacity, global_limit; detail names the wanted host=/os= and the sentence with this machine's names and the satellites) carry reason and since; a new reason logs again with previous and previous_ms. The end of a wait is dispatch.wait_end / dispatch.host_wait_end with waited_for and waited_ms.

Satellite clocks: every satellite report carries the satellite's time. When the difference to the main host changes by more than 0.5 s, the dispatcher logs satellite.clock with host, skew_ms (satellite minus main host), satellite_ts and received_ts (the receive time includes the round trip). gitoodo trace moves the satellite's lines by that skew.

Reading runs#

Both commands only read the log folder, never write, and redact every printed text once more.

  • gitoodo trace <owner/repo#n | run-id> [--local] [--output] prints the timeline of one issue (all its runs and waits) or one run: time, offset from the first line, process (satellite@<host> for remote lines), step, result, duration and details, then a summary per run with the duration of each step, open steps, the total and the longest step. It also asks every configured satellite for its lines of the run (trace_lines request, at most 2000 lines and 4 MiB, newest kept); a satellite that does not answer is named and skipped. --local reads this machine only. It lists the run's worker transcripts; --output prints them.
  • gitoodo diagnose [--hours <n>] summarises the last 24 hours (or n): runs started and their results, per step the count, results and p50/p95/max duration, the most frequent wait reasons, errors by step or event and kind, and the log folder against its budget. A "cleanup" section shows the dispatcher's last repo-hygiene round (#771, <state dir>/hygiene.json): "clean" or the leftovers (host, repo#number or path, branch, reason, unsaved commits and files), the disk guard's reading and the round's errors. Further sections: classifier polls (count, p50/p95/max) with the forge requests of classifier and dispatcher (failures, rate limits, time waited before repeats, worst p95); per service whether it runs now, its last start (version), last end, panics and last warning or error; the size of the worker transcripts. A "questions" section counts the steps of the question flow (see Questions) and lists the issues whose last step is not answer_applied, newest first, with that step and its detail.

The status window's "Run" section (#749) reads the same files for the last 24 hours through the app commands run_overview and run_output (gitoodo_core::logging::overview): the classifier's last poll (time, duration, repos, errors, forge requests and rate limits) with the poll percentiles, the dispatcher's state (running since, current wait and its reason, forge requests, last warning), the newest 20 worker runs with result, failed step and whether a transcript is kept, and the newest 20 question steps. "Show output" opens the end of a run's transcript (at most 256 KiB, from a whole line), redacted line by line once more on reading.

Questions#

Every step of a question is logged with the issue (account/owner/repo#n) and IDs only, never question or answer texts, so "my answer did not land" can be followed step by step:

EventProcessFields (in ctx)
question.postedclassifier, dispatcher, satellitequestion_id, kind, via (classifier intake or worker)
question.post_failedsamequestion_id, kind, via, error (redacted)
question.answer_postedappquestion_id, comment_id, answers (count)
question.answer_failedappquestion_id, category
question.comment_postedappcomment_id
question.answer_seenclassifierquestion_id (intake) or comment_id (detection), via, verdict
question.answer_appliedclassifierquestion_id or comment_id, via, outcome (ready, review, closed, …)
question.answer_pendingclassifiercomment_id, reason: why the newest human reply does not count (once per reason)

answer_pending reasons: detection_off (answer detection is off in the rules), question_open (a question of the block is still unanswered), not_asked (no open question is known), declined (the asked person said no to an approval the intake no longer reads, so the issue keeps waiting). Without a human or a routing rule the login in a card's header is the person asked, so such a reply counts. Comments written by Gitoodo itself are not considered.

Worker transcripts#

Every worker run on the main host or a satellite writes its terminal output to runs/<run_id>/run.log (a restarted run: run.2.log, …). The text is stripped of terminal control sequences, cut at 8 KiB per line and redacted line by line like every log field. Per run at most 2 MiB: the first 256 KiB are kept, then only the two newest tail chunks (run.tail-N.log), with a gap marker in between. When the worker ends the head is compressed to run.log.gz (worker.transcript); a service that starts finishes transcripts a crashed predecessor left (worker.transcripts_recovered). Transcripts count against the log folder budget and age limit like every other file.

A service panic writes one redacted line (and a backtrace only with RUST_BACKTRACE) to its stderr file, never the raw panic text.

On macOS launchd sends a service's stderr to <service>.stderr.log. On Windows the Task Scheduler redirects nothing, so the per-user scheduled task starts gitoodo classifier --service: the CLI removes the flag, treats itself as run by a service manager and points its standard error at the same file before anything else runs (SetStdHandle, no admin rights). A failed redirect is logged as service.stderr_failed.

time                +  process           run           event / step            result        took  detail
10:00:00.000    +0 ms  classifier        -             classifier.ready                            cid=ready-1
10:00:05.000   +5.0 s  dispatcher        r-7k2m9q4xzt  run.start                                   cid=ready-1  host=mini
10:00:05.000   +5.0 s  dispatcher        r-7k2m9q4xzt  queue.wait              ok           5.0 s
10:00:06.210   +6.2 s  dispatcher        r-7k2m9q4xzt  worker.spawn            ok           1.2 s  host=mini
…
10:06:45.000   +6m45s  dispatcher        r-7k2m9q4xzt  run.end                 merged       6m40s

Event types#

EventLevelFields (in ctx)
token_checkinfo / errorhost, forge, category, status (HTTP, if any), missing_scopes / missing_scope
account_add_failederrorerror
sync_starteddebugaccount, host
sync_finishedinfoaccount, host, duration_ms, questions
sync_failedwarnaccount, host, category, error, duration_ms, consecutive_failures, retry_in_ms, repeats
sync_recoveredinfoaccount, host, suppressed_failures
keychain_accessdebug / warn / errorop (get/set/delete), account, result, error (never the content)
settings_changedinfokeys (names of changed settings, never values)
integration_errorwarnintegration (autostart, shortcut, skill, open_url), error, host for open_url
chat_notifyinfo / warnhost, kind, sent, skipped_old, dropped; on failure result, reason (category), repeats (never the URL)
chat_webhook_setinfo (audit)none; chat_webhook_cleared likewise. The URL is never logged
link_sharedinfovia (copy/share), source (question, board, release), host (public forge name, else private; never the URL)
app_restartinfonone; "Restart app" in the help menu

Token check categories: unauthorized, forbidden, not_found, rate_limited, unreachable, tls, server_error, decode, invalid_url. Together with status and host they explain why a token was rejected.

Audit events#

Marked with "audit": true, written on the gitoodo::audit target. They bypass the level filter, so they cannot be switched off in production: account_added, account_removed, account_renewed, token_deleted (with reason).

Throttling#

Identical repeated errors (same key, e.g. the same sync error text) are written once per 10 minutes. The next line that gets through carries repeats, the number of suppressed occurrences; sync_recovered reports suppressed_failures. Failed keychain reads are throttled the same way.

Debugging#

  • Set GITOODO_LOG=debug (overrides the log_level setting) or change the log level in settings.
  • Follow one run: jq 'select(.cid=="sync-1a2b3c")' gitoodo.log
  • One issue across the services: gitoodo trace owner/repo#42, or jq 'select(.issue=="owner/repo#42")' *.jsonl
  • What is slow or failing lately: gitoodo diagnose
  • All problems: jq 'select(.level=="warn" or .level=="error")' gitoodo.log
  • Audit trail: jq 'select(.audit)' gitoodo.log
Docs version: dev