GenAI Utils | Inference Type and Span Creation

[keith-decker]

Description

The GenAI Utils package will include boilerplate and helpers to standardize instrumentation for Generative AI.
This PR provides APIs to minimize the work needed to instrument genai libraries. It provides helpers for starting and stopping LLM invocations, it generates spans aligned with semconvs.

Metrics and Events to come on future PRs

Type of change

• New feature (non-breaking change which adds functionality)

How Has This Been Tested?

• Basic unit test for starting and stopping LLM - Spans
• Unit test for parent/child invocations

Does This PR Require a Core Repo Change?

• Yes. - Link to PR:
• No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

[keith-decker]

A cleaned up version of: #3732

• Flattened LLMInvocation to not use dictionary keys
• Removed the parent run_id, use OTEL context to manage parent/child relationships on spans
• Return the run_id on creation if one is not provided, in order to specify which span to end on finish

[aabmass]

@keith-decker thing I was mentioning in the SIG:
Looks like you already figured out the context manager thing but figured I'd share what I wrote:

opentelemetry-python-contrib/instrumentation-genai/opentelemetry-instrumentation-vertexai/src/opentelemetry/instrumentation/vertexai/patch.py

Lines 101 to 107 in ee947ce

	def _with_instrumentation(
	self,
	instance: client.PredictionServiceClient
	| client_v1beta1.PredictionServiceClient,
	args: Any,
	kwargs: Any,
	):

It's used for both sync and async wrapper, and would work for streaming as well.

[DylanRussell]

Looks good overall. I see one type check error:

/home/runner/work/opentelemetry-python-contrib/opentelemetry-python-contrib/instrumentation-genai/opentelemetry-instrumentation-langchain/src/opentelemetry/instrumentation/langchain/span_manager.py
  /home/runner/work/opentelemetry-python-contrib/opentelemetry-python-contrib/instrumentation-genai/opentelemetry-instrumentation-langchain/src/opentelemetry/instrumentation/langchain/span_manager.py:34:5 - error: Type of "children" is partially unknown
    Type of "children" is "list[Unknown]" (reportUnknownVariableType)

[keith-decker]

Looks good overall. I see one type check error:

/home/runner/work/opentelemetry-python-contrib/opentelemetry-python-contrib/instrumentation-genai/opentelemetry-instrumentation-langchain/src/opentelemetry/instrumentation/langchain/span_manager.py
  /home/runner/work/opentelemetry-python-contrib/opentelemetry-python-contrib/instrumentation-genai/opentelemetry-instrumentation-langchain/src/opentelemetry/instrumentation/langchain/span_manager.py:34:5 - error: Type of "children" is partially unknown
    Type of "children" is "list[Unknown]" (reportUnknownVariableType)

This is part of the langchain instrumentation that just merged to main. Should we open a separate PR to keep it out of the utils PR?

[DylanRussell]

I think typecheck runs against everything that's opted into typecheck, so this check will block/fail on lots of PRs.

It should be fixed in this PR or a separate one..

[aabmass]

I would recommend just setting moving everything from setUpClass into here. Then you don't have to clear any state between tests

[DylanRussell]

This looks good to me now. Just added a few comments

[pmcollins]

Another option here would be to use immutable types, e.g. breaking out LLMInvocation into an LLMRequest and an LLMResponse. It's possible that span and token don't belong in either of these types, but maybe they can be returned in a tuple or a wrapper.