Fixes for the new testgres 1.11.0 #7990

fabriziomello · 2025-04-22T13:26:59Z

A new testgres framework was released that makes PostgresNode::port read-only. Fixed it by properly set the port when initializing a new node.

Failure due to the new testgres version: https://github.com/timescale/timescaledb/actions/runs/14589439220/job/40921093070?pr=7976

References:
https://github.com/postgrespro/testgres/releases/tag/1.11.0
postgrespro/testgres#234

Disable-check: force-changelog-file
Disable-check: approval-count

codecov · 2025-04-22T13:38:24Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.13%. Comparing base (59f50f2) to head (e651aaf).
Report is 1009 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7990      +/-   ##
==========================================
+ Coverage   80.06%   82.13%   +2.06%     
==========================================
  Files         190      250      +60     
  Lines       37181    46284    +9103     
  Branches     9450    11613    +2163     
==========================================
+ Hits        29770    38014    +8244     
- Misses       2997     3668     +671     
- Partials     4414     4602     +188

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

scripts/test_pg_upgrade.py

A new testgres framework was released that makes `PostgresNode::port` read-only. Fixed it by properly set the port when initializing a new node. References: https://github.com/postgrespro/testgres/releases/tag/1.11.0 postgrespro/testgres#234

fabriziomello · 2025-04-22T19:05:13Z

test/sql/updates/post.catalog.sql

+
+-- Functions in schemas:
+--   * _timescaledb_internal
+--   * _timescaledb_functions
+--   * public
+SELECT n.nspname as "Schema",
+  p.proname as "Name",
+  pg_catalog.pg_get_function_result(p.oid) as "Result data type",
+  pg_catalog.pg_get_function_arguments(p.oid) as "Argument data types",
+ CASE p.prokind
+  WHEN 'a' THEN 'agg'
+  WHEN 'w' THEN 'window'
+  WHEN 'p' THEN 'proc'
+  ELSE 'func'
+ END as "Type",
+ CASE
+  WHEN p.provolatile = 'i' THEN 'immutable'
+  WHEN p.provolatile = 's' THEN 'stable'
+  WHEN p.provolatile = 'v' THEN 'volatile'
+ END as "Volatility",
+ CASE
+  WHEN p.proparallel = 'r' THEN 'restricted'
+  WHEN p.proparallel = 's' THEN 'safe'
+  WHEN p.proparallel = 'u' THEN 'unsafe'
+ END as "Parallel",
+ pg_catalog.pg_get_userbyid(p.proowner) as "Owner",
+ CASE WHEN prosecdef THEN 'definer' ELSE 'invoker' END AS "Security",
+ CASE WHEN pg_catalog.array_length(p.proacl, 1) = 0 THEN '(none)' ELSE pg_catalog.array_to_string(p.proacl, E'\n') END AS "Access privileges",
+ l.lanname as "Language",
+ p.prosrc as "Source code",
+ CASE WHEN l.lanname IN ('internal', 'c') THEN p.prosrc END as "Internal name",
+ pg_catalog.obj_description(p.oid, 'pg_proc') as "Description"
+FROM pg_catalog.pg_proc p
+     LEFT JOIN pg_catalog.pg_namespace n ON n.oid = p.pronamespace
+     LEFT JOIN pg_catalog.pg_language l ON l.oid = p.prolang
+WHERE n.nspname OPERATOR(pg_catalog.~) '^(_timescaledb_internal|_timescaledb_functions|public)$' COLLATE pg_catalog.default
+ORDER BY 1, 2, 4;


@melihmutlu added a custom SQL to get the functions output instead of relay on \df+ psql metacommand output that changes over Postgres versions.

svenklemm · 2025-04-22T19:06:53Z

test/sql/updates/post.catalog.sql

+-- Functions in schemas:
+--   * _timescaledb_internal
+--   * _timescaledb_functions
+--   * public
+SELECT n.nspname as "Schema",
+  p.proname as "Name",
+  pg_catalog.pg_get_function_result(p.oid) as "Result data type",
+  pg_catalog.pg_get_function_arguments(p.oid) as "Argument data types",
+ CASE p.prokind
+  WHEN 'a' THEN 'agg'
+  WHEN 'w' THEN 'window'
+  WHEN 'p' THEN 'proc'
+  ELSE 'func'
+ END as "Type",
+ CASE
+  WHEN p.provolatile = 'i' THEN 'immutable'
+  WHEN p.provolatile = 's' THEN 'stable'
+  WHEN p.provolatile = 'v' THEN 'volatile'
+ END as "Volatility",
+ CASE
+  WHEN p.proparallel = 'r' THEN 'restricted'
+  WHEN p.proparallel = 's' THEN 'safe'
+  WHEN p.proparallel = 'u' THEN 'unsafe'
+ END as "Parallel",
+ pg_catalog.pg_get_userbyid(p.proowner) as "Owner",
+ CASE WHEN prosecdef THEN 'definer' ELSE 'invoker' END AS "Security",
+ CASE WHEN pg_catalog.array_length(p.proacl, 1) = 0 THEN '(none)' ELSE pg_catalog.array_to_string(p.proacl, E'\n') END AS "Access privileges",
+ l.lanname as "Language",
+ p.prosrc as "Source code",
+ CASE WHEN l.lanname IN ('internal', 'c') THEN p.prosrc END as "Internal name",
+ pg_catalog.obj_description(p.oid, 'pg_proc') as "Description"
+FROM pg_catalog.pg_proc p
+     LEFT JOIN pg_catalog.pg_namespace n ON n.oid = p.pronamespace
+     LEFT JOIN pg_catalog.pg_language l ON l.oid = p.prolang
+WHERE n.nspname OPERATOR(pg_catalog.~) '^(_timescaledb_internal|_timescaledb_functions|public)$' COLLATE pg_catalog.default
+ORDER BY 1, 2, 4;


Not sure a 30 line sql statement is an improvement over a 4 line psql meta commands

The \df+ output changed so I decided to hardcode this output on the test

i think \df is much easier to maintain, also this seems unrelated to the ci fix so we should probably separate this and get the ci fix in quickly

It is not unrelated because before we ran the psql from the newest version against the old cluster to produce exact the same output and don't have this change. Not PostgresNode::port is read-only so we can't change the port on-the-fly to run psql from newest cluster against oldest. O also think that \df is way easier to maintain but this post.catalog.sql have \df+ since forever. Do u think it is not necessary?

hmm I think \df+ is useful to verify the result of update scripts, if we run the same psql version against both versions there should be no diff, right?

hmm I think \df+ is useful to verify the result of update scripts, if we run the same psql version against both versions there should be no diff, right?

This is exactly what I tried to explain because now the PostgresNode::port is read-only so I can't change it in order to run new psql version against old cluster. Check the .py changes.

fabriziomello added ci force-auto-backport Automatically backport this PR or fix of this issue, even if it's not marked as "bug" labels Apr 22, 2025

fabriziomello self-assigned this Apr 22, 2025

melihmutlu reviewed Apr 22, 2025

View reviewed changes

scripts/test_pg_upgrade.py Outdated Show resolved Hide resolved

fabriziomello force-pushed the fixes_for_new_testgres_version branch 2 times, most recently from 8714ce7 to 595d461 Compare April 22, 2025 18:36

Fixes for the new testgres 1.11.0

e651aaf

A new testgres framework was released that makes `PostgresNode::port` read-only. Fixed it by properly set the port when initializing a new node. References: https://github.com/postgrespro/testgres/releases/tag/1.11.0 postgrespro/testgres#234

fabriziomello force-pushed the fixes_for_new_testgres_version branch from 595d461 to e651aaf Compare April 22, 2025 18:58

fabriziomello commented Apr 22, 2025

View reviewed changes

svenklemm reviewed Apr 22, 2025

View reviewed changes

svenklemm approved these changes Apr 23, 2025

View reviewed changes

fabriziomello merged commit 3e61107 into timescale:main Apr 23, 2025
59 of 60 checks passed

timescale-automation mentioned this pull request Apr 23, 2025

Backport to 2.19.x: #7990: Fixes for the new testgres 1.11.0 #7995

Merged

timescale-automation added the backported-2.19.x label Apr 23, 2025

akuzm added the released-2.20.0 Released in 2.20.0 label May 16, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fixes for the new testgres 1.11.0 #7990

Fixes for the new testgres 1.11.0 #7990

Uh oh!

fabriziomello commented Apr 22, 2025 •

edited

Loading

Uh oh!

codecov bot commented Apr 22, 2025 •

edited

Loading

Uh oh!

Uh oh!

fabriziomello Apr 22, 2025

Uh oh!

svenklemm Apr 22, 2025 •

edited

Loading

Uh oh!

fabriziomello Apr 22, 2025

Uh oh!

svenklemm Apr 22, 2025

Uh oh!

fabriziomello Apr 22, 2025

Uh oh!

svenklemm Apr 22, 2025

Uh oh!

fabriziomello Apr 22, 2025

Uh oh!

Uh oh!

Uh oh!

Fixes for the new testgres 1.11.0 #7990

Fixes for the new testgres 1.11.0 #7990

Uh oh!

Conversation

fabriziomello commented Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

fabriziomello Apr 22, 2025

Choose a reason for hiding this comment

Uh oh!

svenklemm Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fabriziomello Apr 22, 2025

Choose a reason for hiding this comment

Uh oh!

svenklemm Apr 22, 2025

Choose a reason for hiding this comment

Uh oh!

fabriziomello Apr 22, 2025

Choose a reason for hiding this comment

Uh oh!

svenklemm Apr 22, 2025

Choose a reason for hiding this comment

Uh oh!

fabriziomello Apr 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

fabriziomello commented Apr 22, 2025 •

edited

Loading

codecov bot commented Apr 22, 2025 •

edited

Loading

svenklemm Apr 22, 2025 •

edited

Loading