Code

Publish metrics to Dockstore#5198

Code

Open

adamnovak wants to merge 51 commits intomasterDataBiosphere/toil:masterfrom issues/5159-dockstore-metricsDataBiosphere/toil:issues/5159-dockstore-metricsCopy head branch name to clipboard

+2,595-314

Commits

Commits on Dec 5, 2024

• Split workflow retrieval so we can keep the TRS info

  adamnovak

  committed
  cde6b13

  Copy full SHA for cde6b13

• Generate FileNotFoundError for workflows that aren't real

  adamnovak

  committed
  3a8392a

  Copy full SHA for 3a8392a

• Sketch out how we might send some metrics one at a time

  adamnovak

  committed
  189c231

  Copy full SHA for 189c231

Commits on Dec 12, 2024

• Merge remote-tracking branch 'upstream/master' into issues/5159-dockstore-metrics

  adamnovak

  committed
  75415c2

  Copy full SHA for 75415c2

• Refactor to unify Requests calls

  adamnovak

  committed
  dd9592d

  Copy full SHA for dd9592d

• Add more type info for the requests

  adamnovak

  committed
  90a9376

  Copy full SHA for 90a9376

• Record job start times since Dockstore needs them

  adamnovak

  committed
  86d49fd

  Copy full SHA for 86d49fd

• Make worker always save stats

  adamnovak

  committed
  8251860

  Copy full SHA for 8251860

• Refactor Toil home finding

  adamnovak

  committed
  ac7ed25

  Copy full SHA for ac7ed25

• Hook up hooks for recording job and workflow history

  adamnovak

  committed
  1bcb05d

  Copy full SHA for 1bcb05d

Commits on Dec 17, 2024

• Add an intitial attempt at database storage for history

  adamnovak

  committed
  9face90

  Copy full SHA for 9face90

• Fix SQL syntax and semantics

  adamnovak

  committed
  3b9bd3f

  Copy full SHA for 3b9bd3f

Commits on Dec 19, 2024

• Allow submitting workflow attempt metrics with toil history --submit

  adamnovak

  committed
  6817c22

  Copy full SHA for 6817c22

• Fix typing

  adamnovak

  committed
  44043ce

  Copy full SHA for 44043ce

Commits on Jan 9, 2025

• Merge remote-tracking branch 'upstream/master' into issues/5159-dockstore-metrics

  adamnovak

  committed
  8993c62

  Copy full SHA for 8993c62

• Support my-workflows URLs and look for metrics under /api/api

  adamnovak

  committed
  48ed559

  Copy full SHA for 48ed559

• Stop picking a default version when Dockstore has several since we can't get Dockstore's default over TRS

  adamnovak

  committed
  4cd9982

  Copy full SHA for 4cd9982

• Get Dockstore metrics submitted

  adamnovak

  committed
  66fe699

  Copy full SHA for 66fe699

• Log URL to retrieve submitted metrics from

  adamnovak

  committed
  4fbf968

  Copy full SHA for 4fbf968

• Find submittable job attempts in the database

  adamnovak

  committed
  8351101

  Copy full SHA for 8351101

Commits on Jan 10, 2025

• Set up task submission

  adamnovak

  committed
  b44a358

  Copy full SHA for b44a358

• Fix packing code

  adamnovak

  committed
  a1f9084

  Copy full SHA for a1f9084

• Collect stats for failed jobs

  adamnovak

  committed
  d052459

  Copy full SHA for d052459

• Collect and Dockstore-ify task resources

  adamnovak

  committed
  7311967

  Copy full SHA for 7311967

• Actually send new metadata to history manager

  adamnovak

  committed
  b3fb192

  Copy full SHA for b3fb192

• Declare migration bankruptcy

  adamnovak

  committed
  1a56f3a

  Copy full SHA for 1a56f3a

• Fill in the new workflow run extra fields right

  adamnovak

  committed
  cc25e8f

  Copy full SHA for cc25e8f

Commits on Jan 11, 2025

• Implement metrics publication switch and a place to dialog with the user

  adamnovak

  committed
  1888dc4

  Copy full SHA for 1888dc4

Commits on Jan 14, 2025

• Handle Python versions that don't have the autocommit setting yet

  adamnovak

  committed
  a4ef104

  Copy full SHA for a4ef104

• Start on a dialog system that breaks asyncio

  adamnovak

  committed
  7b5eedf

  Copy full SHA for 7b5eedf

• Write plausible tkinter code that thinks about all options

  adamnovak

  committed
  15a72d3

  Copy full SHA for 15a72d3

Commits on Jan 15, 2025

• Implement tkinter dialog

  adamnovak

  committed
  0ae2af6

  Copy full SHA for 0ae2af6

• Give MiniWDL an event loop always

  adamnovak

  committed
  b75aa7c

  Copy full SHA for b75aa7c

• Drop AppleScript as a strategy since its dialogs can't display enough stuff.

  adamnovak

  committed
  e09340e

  Copy full SHA for e09340e

• Improve publishing message

  adamnovak

  committed
  978ee57

  Copy full SHA for 978ee57

• Reword options to handle not having an incognito mode

  adamnovak

  committed
  ac0db21

  Copy full SHA for ac0db21

• Remove toil history command so it can be its own PR

  adamnovak

  committed
  a69a724

  Copy full SHA for a69a724

• Don't actually submit any tasks by default for now

  adamnovak

  committed
  ed6fe90

  Copy full SHA for ed6fe90

• Give the tkinter dialog its own process to avoid linking the main Toil Python to the Mac dock

  adamnovak

  committed
  2ae7fd5

  Copy full SHA for 2ae7fd5

• Document new options and debugging environment variables

  adamnovak

  committed
  cfe151d

  Copy full SHA for cfe151d

• Set database file permissions

  adamnovak

  committed
  9ec3829

  Copy full SHA for 9ec3829

• Merge branch 'master' into issues/5159-dockstore-metrics

  adamnovak

  authored
  2c0578c

  Copy full SHA for 2c0578c

Commits on Jan 17, 2025

• Add apparently-mandatory metavar

  adamnovak

  committed
  6703d53

  Copy full SHA for 6703d53

• Merge remote-tracking branch 'upstream/issues/5159-dockstore-metrics' into issues/5159-dockstore-metrics

  adamnovak

  committed
  6ae57c3

  Copy full SHA for 6ae57c3

• Merge remote-tracking branch 'upstream/master' into issues/5159-dockstore-metrics

  adamnovak

  committed
  e620d72

  Copy full SHA for e620d72

• Make tests and code agree on missing version behavior

  adamnovak

  committed
  e3e6414

  Copy full SHA for e3e6414

• Liberally festoon DB code with closes and adopt infinite retries from caching code

  adamnovak

  committed
  fbdd58d

  Copy full SHA for fbdd58d

Commits on Jan 21, 2025

• Merge remote-tracking branch 'upstream/master' into issues/5159-dockstore-metrics

  adamnovak

  committed
  8b711f1

  Copy full SHA for 8b711f1

• Move database retries to top-level history methods

  adamnovak

  committed
  2b91421

  Copy full SHA for 2b91421

Commits on Jan 22, 2025

• Handle workflow metadata not being available on restart in CWL

  adamnovak

  committed
  4076bd7

  Copy full SHA for 4076bd7

Commits on Jan 23, 2025

• Add version to TRS ID test

  adamnovak

  committed
  ab806ee

  Copy full SHA for ab806ee