Python SDK 2.4.0 (finance_research, research background mode, source_control, output_schema) - #12
Python SDK 2.4.0 (finance_research, research background mode, source_control, output_schema)#12tyler5673 wants to merge 1 commit into
Conversation
bd33525 to
a17ae2a
Compare
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| ```python | ||
| res = you.research( | ||
| input="Are Acme Logistics DE and Acme Logistics NJ the same entity?", | ||
| output_schema={ | ||
| "type": "object", | ||
| "properties": { | ||
| "same_entity": {"type": "boolean"}, | ||
| "confidence": {"type": "number"}, | ||
| "evidence": {"type": "array", "items": {"type": "string"}}, | ||
| }, | ||
| "required": ["same_entity", "confidence", "evidence"], | ||
| }, | ||
| ) | ||
| assert res.output.content_type.value == "object" | ||
| # `content` is now a dict, not a str | ||
| verdict = res.output.content # type: dict | ||
| print(verdict["same_entity"]) | ||
| ``` | ||
|
|
||
| Code that does `res.output.content.lower()` or similar string-only operations will still work for typical text responses (the value remains a `str`), but if you opt into `output_schema` you must branch on `content_type` before calling string methods. |
There was a problem hiding this comment.
This output_schema example (and the trailing "Code that does res.output.content.lower()..." paragraph) is placed under the "Environment variable renamed to YDC_API_KEY" heading, but it documents structured output — it belongs under "#### Research output.content is now Union[str, object]" (line 75), which currently has only prose and no example. As written, the env-var migration section confusingly ends with an unrelated structured-output snippet. Please move this block up under the output.content heading.
64fa89c to
7443a06
Compare
| results = you.search.unified( | ||
| query="latest advances in fusion energy research", | ||
| count=5, | ||
| boost_domains=["nature.com", "science.org", "arxiv.org"], | ||
| ) |
There was a problem hiding this comment.
search.unified (GET /v1/search) types boost_domains as Optional[str] — a single comma-separated value (see search.py:29 and SearchRequest.boost_domains: Optional[str] in models/searchop.py:129). Passing a list here builds SearchRequest(boost_domains=[...]), which raises a pydantic ValidationError at request time.
Either pass a comma-separated string, or use the POST variant you.search_post(..., boost_domains=[...]), whose SearchRequestBody.boost_domains is Optional[List[str]].
| results = you.search.unified( | |
| query="latest advances in fusion energy research", | |
| count=5, | |
| boost_domains=["nature.com", "science.org", "arxiv.org"], | |
| ) | |
| results = you.search.unified( | |
| query="latest advances in fusion energy research", | |
| count=5, | |
| boost_domains="nature.com,science.org,arxiv.org", | |
| ) |
c355d90 to
1ce3dac
Compare
| # Stream mode: poll until terminal event OR deadline, then fetch the | ||
| # final detail. Track elapsed time against the user's ``timeout_s`` so | ||
| # a stuck stream can't block forever. | ||
| for evt in stream_research_events_raw(client, task.task_id): | ||
| name = evt.event | ||
| if name in {"response.done", "complete"}: | ||
| return poll_research_task( | ||
| client, | ||
| task.task_id, | ||
| interval_s=interval_s, | ||
| timeout_s=max(interval_s, deadline - time.monotonic()), | ||
| ) | ||
| if name in {"error", "cancelled"}: | ||
| raise RuntimeError( | ||
| f"research task {task.task_id} ended in non-completed state: {name}" | ||
| ) | ||
| if time.monotonic() >= deadline: | ||
| raise TimeoutError( | ||
| f"research task {task.task_id} did not complete within {timeout_s}s" | ||
| ) |
There was a problem hiding this comment.
The deadline check (line 260) only runs after an event is received. If the server holds the SSE connection open but stops emitting events — or emits only keepalive comment lines (:...), which _parse_event drops with data_required=False and never yields — the for evt in ... loop blocks on the network read and the timeout_s deadline is never evaluated. In that case only httpx's per-read timeout (timeout_ms, reset by each keepalive byte) bounds the wait, not timeout_s.
So the comment "a stuck stream can't block forever" overstates the guarantee: timeout_s is enforced only while terminal-less events keep flowing. Consider either documenting that timeout_s bounds inter-event progress (not wall-clock on an idle socket), or enforcing wall-clock deadline independently of event arrival (e.g. a watchdog / read timeout derived from remaining budget). Same pattern in research_and_wait_async (lines 296–312).
| async for evt in stream_research_events_raw_async(client, task.task_id): | ||
| name = evt.event | ||
| if name in {"response.done", "complete"}: | ||
| return await poll_research_task_async( | ||
| client, | ||
| task.task_id, | ||
| interval_s=interval_s, | ||
| timeout_s=max(interval_s, deadline - time.monotonic()), | ||
| ) | ||
| if name in {"error", "cancelled"}: | ||
| raise RuntimeError( | ||
| f"research task {task.task_id} ended in non-completed state: {name}" | ||
| ) | ||
| if time.monotonic() >= deadline: | ||
| raise TimeoutError( | ||
| f"research task {task.task_id} did not complete within {timeout_s}s" | ||
| ) |
There was a problem hiding this comment.
Minor resource-cleanup nuance: when this async for returns early on a terminal event (line 299), the outer async generator stream_research_events_raw_async is left suspended. Its contextlib.aclosing(...) cleanup (which closes the underlying httpx SSE response) only fires when the generator is aclose()d — which here happens via GC-driven finalization, exactly the non-deterministic path the module docstring/comment (lines 479–481) warns against. The sync version is fine (refcount closes it promptly), but the async early-return can leak the connection until GC. Wrapping the iteration in contextlib.aclosing(stream_research_events_raw_async(...)) here would make closure deterministic.
| class Content(BaseModel): | ||
| pass |
There was a problem hiding this comment.
Root cause of the biggest usability gap in this release (well-documented in CHANGELOG/MIGRATION/tests, flagging for visibility): Content is a field-less model and BaseModel uses pydantic's default extra="ignore", so when content_type="object" the server's structured payload is silently dropped and output.content becomes an empty Content(). That means the headline output_schema feature returns nothing usable through the typed path — callers must re-issue a synchronous call to recover it (per MIGRATION). Same root cause makes TaskDetail.result empty for background research.
This is generated from the oneOf: [object, string] in the spec, so the durable fix belongs in overlays/python_overlay.yaml: give the object branch additionalProperties: true (or x-speakeasy-type/free-form map) so it generates as Dict[str, Any] and the structured content survives unmarshaling. Worth doing before promoting output_schema out of beta, since today the feature is effectively non-functional via the SDK.
Review: 2.4.0 — Finance Research, background/streaming research, output_schema/source_control, env-var renameFocused on the manually-maintained surface (tests, CHANGELOG, MIGRATION, overlay, pyproject, and the hand-maintained research_helpers.py / hooks / security.py). Overall this is a high-quality, unusually well-documented PR — CHANGELOG and MIGRATION are candid about limitations, hand-edits are explicitly flagged as regen-fragile, and test coverage for the new helpers is thorough (sync + async, success/timeout/failed, poll + stream, tolerant SSE decode, UA hook). Nice work. Worth addressing before promoting
|
| "nytimes.com", | ||
| "wired.com", | ||
| ], crawl_timeout=10, | ||
| RetryConfig("backoff", BackoffStrategy(1, 50, 1.1, 100), False)) |
There was a problem hiding this comment.
This generated example is not valid Python — RetryConfig(...) is a positional argument appearing after keyword arguments (crawl_timeout=10,), which is a SyntaxError. It should be passed as the retries= keyword. Since README is Speakeasy-generated, the real fix likely belongs in the overlay/spec (the per-operation example), but the rendered doc is currently broken.
| RetryConfig("backoff", BackoffStrategy(1, 50, 1.1, 100), False)) | |
| crawl_timeout=10, | |
| retries=RetryConfig("backoff", BackoffStrategy(1, 50, 1.1, 100), False)) |
| # Stream mode: poll until terminal event OR deadline, then fetch the | ||
| # final detail. Track elapsed time against the user's ``timeout_s`` so | ||
| # a stuck stream can't block forever. | ||
| for evt in stream_research_events_raw(client, task.task_id): | ||
| name = evt.event | ||
| if name in {"response.done", "complete"}: | ||
| return poll_research_task( | ||
| client, | ||
| task.task_id, | ||
| interval_s=interval_s, | ||
| timeout_s=max(interval_s, deadline - time.monotonic()), | ||
| ) | ||
| if name in {"error", "cancelled"}: | ||
| raise RuntimeError( | ||
| f"research task {task.task_id} ended in non-completed state: {name}" | ||
| ) | ||
| if time.monotonic() >= deadline: | ||
| raise TimeoutError( | ||
| f"research task {task.task_id} did not complete within {timeout_s}s" | ||
| ) |
There was a problem hiding this comment.
The comment says the elapsed-time tracking means "a stuck stream can't block forever," but the deadline check only runs after an event is received. A stream that stays connected but emits nothing (or stalls mid-workflow) blocks in for evt in stream_research_events_raw(...) on the socket read and is bounded only by the httpx read timeout — not by timeout_s. Consider clarifying the comment, or enforcing timeout_s independently of event arrival (e.g. a wall-clock guard around the read). Same pattern in research_and_wait_async.
|
PR Review: 2.4.0 — Finance Research, background/stream research, YDC_API_KEY Reviewed with the understanding that src/youdotcom/ is largely Speakeasy-generated; focused on the hand-maintained surface (tests/, research_helpers.py, utils/security.py, CHANGELOG.md, MIGRATION.md, docs, pyproject.toml). Overall. Solid, well-documented release. The env-var precedence change is clean and tested, the hand-maintained research_helpers module is coherent, and the docs (CHANGELOG/MIGRATION) are unusually thorough — including honest callouts of the extra="ignore" limitation that leaves TaskDetail.result / structured output.content untyped. Test coverage for the new functionality is strong: env precedence (primary/fallback/none/explicit), poll terminal + timeout + failed states (sync & async), tolerant SSE streaming with unknown event names, the User-Agent override hook, finance_research, and the output_schema/source_control beta params. Mockserver fixtures line up with every test name. Findings (all posted inline):
Security. No hardcoded secrets, injection, or unsafe patterns introduced. The YDC_API_KEY -> YOU_API_KEY_AUTH fallback is reasonable and covered by tests/test_security_env.py. Test API keys are placeholder literals. None of the above are hard blockers except the version reconciliation (item 2) and the SemVer decision (item 1), which are worth resolving before release. |
Adds Finance Research API, Research background mode (GET poll / SSE stream), Research source_control and output_schema, Search boost_domains, and Contents max_age. Breaking (with documented migration): - New 'FinanceResearchEffort' enum (DEEP, EXHAUSTIVE) joins existing 'ResearchEffort'. Both names preserved cleanly via named-component refs in the OpenAPI specs (no 'ResearchResearchEffort' doubling). - livecrawl_formats now strictly typed as Optional[List[LiveCrawlFormats]]. - 'you.research()' response is now 'Union[ResearchResponse, TaskResponse]' (TaskResponse returned when background=True). - 'output.content' is now 'Union[str, object]' for output_schema responses. - Shared 401/403/422 error shapes consolidated to named '*ResponseError' classes (per-endpoint classes still raised). 'workflow.yaml' references 'https://you.com/specs/openapi_finance_research.yaml' even though that URL is not yet live (frontend PR youdotcom #12477 is unmerged). A regen from public URLs is therefore expected to produce a diff against this commit until #12477 lands. The SDK source under 'src/' contains all 2.4.0 work as of this commit. Two spec-side fixes on #12477 make this regen match cleanly when it lands: - 'ResearchEffort' and 'FinanceResearchEffort' promoted to named components (avoids Speakeasy disambiguator doubling the names). - 'ResearchTaskStreamEvent.event' enum widened to include the server's synthetic fallback names ('completed', 'failed') for stale RabbitMQ streams, in addition to the documented 'connected', 'response.done', 'complete', 'error', 'cancelled'. Post-generation work (hand-maintained, not regenerated): - src/youdotcom/research_helpers.py adds research_background[_async], poll_research_task[_async], research_and_wait[_async], and tolerant stream_research_events_raw[_async] helpers for background-mode workflows. - src/youdotcom/_hooks/registration.py's YDCUserAgentOverrideHook now respects custom 'sdk_configuration.user_agent' instead of unconditionally rewriting the header. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
| x-speakeasy-name-override: create | ||
| # Allow extras on the anonymous object branch of `output.content` so | ||
| # `output_schema=` requests preserve the structured payload through SDK | ||
| # unmarshal (today: `class Content(BaseModel): pass` drops everything | ||
| # due to pydantic `extra="ignore"`). Reviewed as part of Step 4i-9. | ||
| - target: $["components"]["schemas"]["ResearchResponse"]["properties"]["output"]["properties"]["content"]["oneOf"][1] | ||
| update: | ||
| additionalProperties: true | ||
| # Same fix for `TaskDetail.result` so background-mode research surfaces | ||
| # the typed `ResearchResponse` payload (today: `class Result(BaseModel): pass` | ||
| # round-trips as an empty dict). | ||
| - target: $["components"]["schemas"]["TaskDetail"]["properties"]["result"] | ||
| update: | ||
| additionalProperties: true |
There was a problem hiding this comment.
These two new overlay actions are not applied in this PR — they're a no-op as checked in.
I verified against the regenerated artifacts:
.speakeasy/out.openapi.yamlstill hasResearchResponse.output.content.oneOf[1]as bare{type: object}andTaskDetail.resultas{type: object, nullable: true}— noadditionalProperties: trueon either.- The generated models are unchanged:
class Content(BaseModel): passandclass Result(BaseModel): pass(both inherit pydantic's defaultextra="ignore").
The original v1.0.0 overlay actions (x-speakeasy-name-override) are present in out.openapi.yaml, so the compiled spec is post-overlay — it was just regenerated against the overlay before these two actions were added (overlay bumped to 1.1.0 but no subsequent regen).
Net effect: the structured-output / TaskDetail.result payload-dropping bug these actions target is still live, exactly as the CHANGELOG and MIGRATION caveats describe. Before merge, please either (a) regenerate so the fix lands — and then update the CHANGELOG "Hand-maintained additions" + MIGRATION output_schema caveats that currently say it's broken, or (b) drop these actions so the overlay doesn't imply a fix that isn't shipped. Worth also confirming the JSONPath actually matches after regen (the oneOf[1] index in particular).
| res = you.search_post(query="What are the latest geopolitical updates from India", count=10, language=models.Language.EN, exclude_domains=[ | ||
| "spam-site.com", | ||
| "other-site.com", | ||
| ], boost_domains=[ | ||
| "nytimes.com", | ||
| "wired.com", | ||
| ], crawl_timeout=10, | ||
| RetryConfig("backoff", BackoffStrategy(1, 50, 1.1, 100), False)) |
There was a problem hiding this comment.
This generated snippet is invalid Python: RetryConfig(...) is a positional argument placed after keyword arguments (crawl_timeout=10, RetryConfig(...)), which is a SyntaxError. It should be retries=RetryConfig(...).
This is a Speakeasy code-sample generation artifact (the per-method example changed from research to search_post), not a hand edit — but it ships in the README, so worth being aware of. A regen with the overlay fix (see the overlay comment) may also refresh this; otherwise consider whether the example operation choice can be pinned.
| interval_s=interval_s, | ||
| timeout_s=timeout_s, | ||
| ) | ||
| # Stream mode: poll until terminal event OR deadline, then fetch the |
There was a problem hiding this comment.
Minor: the comment says "so a stuck stream can't block forever," but the deadline is only checked after each event is received. If the SSE connection stays open but emits no events, the for evt in ... loop blocks on the socket read up to the httpx client timeout, not timeout_s. The timeout guarantee only holds while events keep arriving. Consider softening the comment, or wrapping the stream read with a real deadline if a hard timeout_s bound matters here.
Review summary — v2.4.0 (Finance Research + background research + SSE)Overall this is a solid, well-documented release. The hand-maintained pieces (tests, CHANGELOG, MIGRATION, overlay, env-var precedence, UA hook) are thoughtful and the CHANGELOG/MIGRATION are unusually honest about current limitations. Test coverage for the new surface is strong. One issue should be resolved before merge; the rest are minor. 🔴 Should fix before merge1. The overlay's 🟡 Minor / nits
✅ Verified good
Note: I couldn't execute the test suite in this environment (sandboxed), so the test verification above is by inspection of the fixtures/handlers, which line up with the referenced |
1 participant
Summary
Added
you.finance_research(...)over a finance-optimized index (SEC filings, transcripts, analyst coverage, news). Onlydeepandexhaustiveefforts.you.research(..., background=True)returns aTaskResponse. Submit without blocking.you.get_research_task(task_id)returns aTaskDetail;you.stream_research_task(task_id)returns a typed SSEEventStream.source_control: optionalinclude_domains/exclude_domains/boost_domains/freshness/country.output_schema: structured JSON output, withoutput.contenttypedUnion[str, object].boost_domains: ranks BUT does NOT restrict (max 500 domains).max_age: int, nullable, seconds, default null.src/youdotcom/research_helpers.py(NOT touched by Speakeasy regen):research_background[_async]— typedTaskResponse, drop-in for narrow.poll_research_task[_async]— wait until terminal state.research_and_wait[_async]— submit+wait (poll or stream), returns finalTaskDetail.stream_research_events_raw[_async]— tolerant SSE iterator that accepts non-enum event names (research.searching, etc.) and returnsRawStreamEvent.sdk_configuration.user_agentso integrations (langchain-youdotcom, youdotcom-temporal, n8n-nodes-youdotcom) can identify themselves with simplyclient.sdk_configuration.user_agent = "<integration>/<version>"instead of swapping hooks.Changed
Research200 isUnion[ResearchResponse, TaskResponse].Research.output.contentisUnion[str, object]foroutput_schemaresponses.ResearchEffort→ResearchResearchEffortrename (collision withFinanceResearchResearchEffort).livecrawl_formatsstrictlyOptional[List[LiveCrawlFormats]].401/403/422errors consolidated toUnauthorizedResponseError/ForbiddenResponseError/UnprocessableEntityResponseError. Per-endpoint classes (ResearchUnauthorizedError,FinanceResearchUnprocessableEntityError, etc.) still raised.Tests: 98 mock/perf tests pass against
tests/mockserveron :18080. 15 live tests cleanly skip (noYOU_API_KEY_AUTH).Checklist
speakeasy runagainst local spec overrides for PLT-1928 spec drift pass).pyproject.toml,gen.yaml,_version.py(all 2.4.0).## [2.4.0] - 2026-07-09).## 2.3.0 → 2.4.0 (Latest));## 1.x → 2.3.0retained./v1/finance_research, POST/v1/researchbackgroundcase, GET/v1/research/{task_id}, GET/v1/research/{task_id}/stream).Migration notes for downstream users
See
MIGRATION.md. Key points:Outstanding at merge time
youdotcom#12477 (PLT-1928) is draft, unmerged at the time of authoring this PR. It introduces the canonicalopenapi_finance_research.yamlspec and the background-mode + SSE surface this SDK depends on. If #12477 is not merged before this PR is merged, releases publishing againsthttps://you.com/specs/...public URLs will produce a diff against this commit (the public URLs will still serve the older 5 specs). The SDK source undersrc/already contains all 2.4.0 work.src/youdotcom/research_helpers.py,src/youdotcom/_hooks/registration.py,tests/test_research_helpers.py) should NOT vanish in that diff.tests/test_live.pyruns against the real API and is skipped withoutYOU_API_KEY_AUTH. Recommended smoke run before merging:export YOU_API_KEY_AUTH="..." && .venv/bin/python -m pytest tests/test_live.py -v. Long efforts (DEEP,EXHAUSTIVE) may exceed default pytest timeout — pass--timeout=600.