Skip to content

Commit 71b1f07

Browse files
erikamoverikamov
committed
Extract exposures files (questions/cards and dashboards) from Metabase.
1 parent 9cedf1c commit 71b1f07

File tree

2,061 files changed

+56049
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

2,061 files changed

+56049
-0
lines changed

‎warehouse/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -159,6 +159,7 @@ Some additional helpful commands:
159159
- `poetry run dbt compile` -- will compile all the models (generate SQL, with references resolved) but won't execute anything in the warehouse; useful for visualizing what dbt will actually execute
160160
- `poetry run dbt docs generate` -- will generate the dbt documentation
161161
- `poetry run dbt docs serve` -- will "serve" the dbt docs locally so you can access them via `http://localhost:8080`; note that you must `docs generate` before you can `docs serve`
162+
- `poetry run dbt-metabase exposures --output-grouping="type" --output-path=models/metabase/ --manifest-path=target/manifest.json --metabase-url https://dashboards.calitp.org --metabase-api-key="INFORM_SERVICE_KEY"` -- will create/update files for questions (cards) and dashboards extracted from Metabase that can be visualized in dbt documentation. You can search for specific models to see where they are refereced in Metabase. For more details see [exposures documentation](https://github.com/gouline/dbt-metabase?tab=readme-ov-file#exposure-extraction).
162163
163164
### Incremental model considerations
164165

‎warehouse/models/metabase/card/1.yml‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
version: 2
2+
exposures:
3+
- name: gtfs_schedule_feed_daily_download_status
4+
label: GTFS schedule feed daily download status
5+
description: '### Visualization: Bar
6+
7+
8+
No description provided in Metabase
9+
10+
11+
#### Query
12+
13+
14+
```
15+
16+
WITH trunc_status AS (SELECT LEFT(status, 30) AS `status` FROM `views.gtfs_status_latest`)
17+
18+
SELECT status, count(*) AS `count`
19+
20+
FROM trunc_status
21+
22+
GROUP BY `status`
23+
24+
ORDER BY `status` ASC
25+
26+
```
27+
28+
29+
#### Metadata
30+
31+
32+
Metabase ID: __1__
33+
34+
35+
Created On: __2021-06-11T19:56:13.905835Z__'
36+
type: analysis
37+
url: https://dashboards.calitp.org/card/1
38+
maturity: medium
39+
owner:
40+
name: Administrator of Metabase
41+
email: [email protected]
42+
depends_on: []
43+
meta:
44+
last_used_at: '2024-09-20T17:54:54.131902Z'

‎warehouse/models/metabase/card/10.yml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
version: 2
2+
exposures:
3+
- name: validator_error_differences_from_prev_day
4+
label: Validator error differences from prev day
5+
description: "### Visualization: Bar\n\nNo description provided in Metabase\n\n\
6+
#### Query\n\n```\nWITH daily_error_counts AS (\nSELECT `views.validation_code_metrics`.`metric_date`\
7+
\ AS `metric_date`, `views.validation_code_metrics`.`calitp_itp_id` AS `calitp_itp_id`,\
8+
\ `views.validation_code_metrics`.`calitp_url_number` AS `calitp_url_number`,\
9+
\ sum(`views.validation_code_metrics`.`n_notices`) AS ttl_notices\nFROM `views.validation_code_metrics`\n\
10+
WHERE `views.validation_code_metrics`.`severity` = 'ERROR'\nGROUP BY metric_date,\
11+
\ `calitp_itp_id`, `calitp_url_number`\nORDER BY metric_date ASC, `calitp_itp_id`\
12+
\ ASC, `calitp_url_number` ASC\n),\ndaily_lagged AS(\nSELECT \n *\n , LAG(ttl_notices)\
13+
\ OVER (PARTITION BY calitp_itp_id, calitp_url_number ORDER BY metric_date)\
14+
\ AS ttl_notices_prev\nFROM daily_error_counts\n)\nSELECT \n *\n , ttl_notices\
15+
\ - ttl_notices_prev AS ttl_notices_diff\nFROM daily_lagged\n```\n\n#### Metadata\n\
16+
\nMetabase ID: __10__\n\nCreated On: __2021-06-21T15:57:33.314299Z__"
17+
type: analysis
18+
url: https://dashboards.calitp.org/card/10
19+
maturity: medium
20+
owner:
21+
name: Michael Chow
22+
email: [email protected]
23+
depends_on: []
24+
meta:
25+
last_used_at: '2024-09-20T17:54:54.131902Z'

‎warehouse/models/metabase/card/100.yml‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
version: 2
2+
exposures:
3+
- name: validation____daily_errors
4+
label: Validation || Daily Errors
5+
description: '### Visualization: Bar
6+
7+
8+
No description provided in Metabase
9+
10+
11+
#### Metadata
12+
13+
14+
Metabase ID: __100__
15+
16+
17+
Created On: __2021-08-26T15:39:22.530763Z__'
18+
type: analysis
19+
url: https://dashboards.calitp.org/card/100
20+
maturity: medium
21+
owner:
22+
name: Natalya Diaz
23+
email: [email protected]
24+
depends_on: []
25+
meta:
26+
last_used_at: '2024-09-20T17:54:54.131902Z'

‎warehouse/models/metabase/card/103.yml‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
version: 2
2+
exposures:
3+
- name: validation____vender_distinct_feeds_by_errors____90_days
4+
label: Validation || Vender Distinct Feeds by Errors || 90 days
5+
description: '### Visualization: Bar
6+
7+
8+
Filtered severity is error, N notices is greater than 1
9+
10+
11+
#### Metadata
12+
13+
14+
Metabase ID: __103__
15+
16+
17+
Created On: __2021-08-26T16:01:59.859545Z__'
18+
type: analysis
19+
url: https://dashboards.calitp.org/card/103
20+
maturity: medium
21+
owner:
22+
name: Natalya Diaz
23+
email: [email protected]
24+
depends_on: []
25+
meta:
26+
last_used_at: '2024-09-20T17:54:54.131902Z'

‎warehouse/models/metabase/card/104.yml‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
version: 2
2+
exposures:
3+
- name: validation___validation_errors_resolved
4+
label: Validation|| Validation Errors Resolved
5+
description: "### Visualization: Line\n\nNo description provided in Metabase\n\
6+
\n#### Query\n\n```\nWITH \ndaily_codes_enh AS (\n SELECT date_trunc(`views.validation_fact_daily_feed_codes`.`date`,\
7+
\ day) \n AS `date`, count(distinct `views.validation_fact_daily_feed_codes`.`code`)\
8+
\ AS `validation_errors`\n FROM `views.validation_fact_daily_feed_codes`\n\
9+
\ LEFT JOIN `views.validation_dim_codes` `validation_dim_codes__via__cod`\
10+
\ \n ON `views.validation_fact_daily_feed_codes`.`code` = `validation_dim_codes__via__cod`.`code`\n\
11+
\ WHERE (`views.validation_fact_daily_feed_codes`.`n_notices` > 0 AND `validation_dim_codes__via__cod`.`severity`\
12+
\ = 'ERROR')\n GROUP BY date\n),\ndaily_code_count AS (\nSelect *, LAG (validation_errors,\
13+
\ 1 )OVER (ORDER BY date) as validation_errors_previous_day\nfrom daily_codes_enh\n\
14+
)\nSelect * , (select validation_errors_previous_day - validation_errors) as\
15+
\ validation_errors_resolved \nfrom daily_code_count \n```\n\n#### Metadata\n\
16+
\nMetabase ID: __104__\n\nCreated On: __2021-08-26T21:32:56.887542Z__"
17+
type: analysis
18+
url: https://dashboards.calitp.org/card/104
19+
maturity: medium
20+
owner:
21+
name: Natalya Diaz
22+
email: [email protected]
23+
depends_on: []
24+
meta:
25+
average_query_time: '0:01.127'
26+
last_used_at: '2024-11-05T21:43:33.056178Z'

‎warehouse/models/metabase/card/105.yml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
version: 2
2+
exposures:
3+
- name: validation____errors_introduced____daily__drilldown
4+
label: Validation || Errors Introduced || Daily Drilldown
5+
description: '### Visualization: Bar
6+
7+
8+
Across all feeds, which codes didn''t have errors on the previous day, but do
9+
today? Allows clicking through to see which agencies produced the error codes.
10+
11+
12+
#### Metadata
13+
14+
15+
Metabase ID: __105__
16+
17+
18+
Created On: __2021-08-27T14:56:09.509032Z__'
19+
type: analysis
20+
url: https://dashboards.calitp.org/card/105
21+
maturity: medium
22+
owner:
23+
name: Michael Chow
24+
email: [email protected]
25+
depends_on: []
26+
meta:
27+
last_used_at: '2024-09-20T17:54:54.131902Z'

‎warehouse/models/metabase/card/106.yml‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
version: 2
2+
exposures:
3+
- name: summary____feed_start_date
4+
label: Summary || Feed Start Date
5+
description: '### Visualization: Scalar
6+
7+
8+
No description provided in Metabase
9+
10+
11+
#### Metadata
12+
13+
14+
Metabase ID: __106__
15+
16+
17+
Created On: __2021-09-01T14:46:39.175614Z__'
18+
type: analysis
19+
url: https://dashboards.calitp.org/card/106
20+
maturity: medium
21+
owner:
22+
name: Michael Chow
23+
email: [email protected]
24+
depends_on: []
25+
meta:
26+
last_used_at: '2024-09-20T17:54:54.131902Z'

‎warehouse/models/metabase/card/107.yml‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
version: 2
2+
exposures:
3+
- name: summary____feed_end_date
4+
label: Summary || Feed End Date
5+
description: '### Visualization: Scalar
6+
7+
8+
No description provided in Metabase
9+
10+
11+
#### Metadata
12+
13+
14+
Metabase ID: __107__
15+
16+
17+
Created On: __2021-09-01T14:49:05.740651Z__'
18+
type: analysis
19+
url: https://dashboards.calitp.org/card/107
20+
maturity: medium
21+
owner:
22+
name: Michael Chow
23+
email: [email protected]
24+
depends_on: []
25+
meta:
26+
last_used_at: '2024-09-20T17:54:54.131902Z'

‎warehouse/models/metabase/card/108.yml‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
version: 2
2+
exposures:
3+
- name: summary____feed_version
4+
label: Summary || Feed Version
5+
description: '### Visualization: Scalar
6+
7+
8+
No description provided in Metabase
9+
10+
11+
#### Metadata
12+
13+
14+
Metabase ID: __108__
15+
16+
17+
Created On: __2021-09-01T14:52:20.0063Z__'
18+
type: analysis
19+
url: https://dashboards.calitp.org/card/108
20+
maturity: medium
21+
owner:
22+
name: Michael Chow
23+
email: [email protected]
24+
depends_on: []
25+
meta:
26+
last_used_at: '2024-09-20T17:54:54.131902Z'

0 commit comments

Comments
 (0)