data : add new API ELT pipeline data engineering project

[Harshan07-web]

Summary

Added a new data engineering project which fetches data from public api, then normalizes the JSON data, converts them to a pandas dataframe, handles missing and duplicate values and the exports the CSV file. This CSV file can further be used in data analytics projects.

Related Issue

Closes #147

Type of Change [required]

• Bug fix — resolves a broken behaviour
• Feature — adds new functionality
• Data — adds new projects to data/projects.json
• Documentation — updates docs, README, or code comments only
• Style — CSS or visual changes only, no logic change
• Refactor — restructures code without changing behaviour
• Test — adds or updates tests

What Was Changed

File	Change made
data/projects.json	Added new project id 9 API ETL Pipeline
startercode/api_data_pipeline.py	Added a sample code skeleton with project objective and steps ONLY

How to Test This PR

1. Clone this branch: git checkout your-branch-name
2. Install dependencies: pip install -r requirements.txt
3. Run the app: python app.py
4. Open http://127.0.0.1:5000 and select skills python,pandas
5. select Experience level intermediate
6. Area of interest Data
7. Run the tests: python tests/test_basic.py

Expected test output:

27 passed, 0 failed out of 27 tests

Test Results

 PASS  test_projects_json_loads
  PASS  test_each_project_has_required_fields
  PASS  test_find_project_by_id_found
  PASS  test_find_project_by_id_missing
  PASS  test_parse_skills_basic
  PASS  test_parse_skills_empty_string
  PASS  test_parse_skills_single_entry
  PASS  test_score_single_project_full_match
  PASS  test_score_single_project_no_match
  PASS  test_get_recommendations_returns_results
  PASS  test_get_recommendations_max_three
  PASS  test_get_recommendations_no_match_returns_empty
  PASS  test_get_recommendations_result_format
  PASS  test_validate_all_valid
  PASS  test_validate_missing_skills
  PASS  test_validate_missing_level
  PASS  test_validate_missing_interest
  PASS  test_validate_missing_time
  PASS  test_validate_all_missing
  PASS  test_home_route
  PASS  test_recommend_api_valid
  PASS  test_recommend_api_missing_field
  PASS  test_recommend_api_empty_body
  PASS  test_project_detail_found
  PASS  test_project_detail_not_found
  PASS  test_view_code_found
  PASS  test_download_code_found

Screenshots (if UI change)

Before	After
screenshot	screenshot

Self-Review Checklist [required]

• I have read CONTRIBUTING.md and followed all guidelines
• My branch name follows the convention: feat/, fix/, docs/, data/, style/, test/
• I have run python tests/test_basic.py and all 27 tests pass
• I have run flake8 . locally and there are no errors
• I have not introduced any print() or console.log() debug statements
• Every new function I wrote has a docstring
• I have not modified files outside the scope of the linked issue
• If I changed the UI, I tested it at 375px (mobile) and 1280px (desktop)
• If I added a project to the dataset, it has all required JSON fields

Notes for Reviewer

Please note that the sample code for this project is not fully written. Only the skeleton of the sample code is created(comments), after the project is approved, I'll be happy to work on the complete sample code.

[vercel]

@Harshan07-web is attempting to deploy a commit to the komalsony234-1530's projects Team on Vercel.

A member of the Team first needs to authorize it.

[github-actions]

Thank you for submitting your first pull request to DevPath.

Before review:

• Complete the PR template fully
• Ensure all tests pass
• Link your PR to an issue
• Keep changes scoped to the issue

A maintainer will review your contribution soon.

[komalharshita]

Good contribution overall — this is a meaningful addition to the DevPath dataset and introduces a stronger intermediate-level data engineering project compared to typical beginner project submissions.

What’s done well:

• the project idea is relevant and industry-aligned,
• the roadmap progression is structured logically,
• the learning goals are clearly explained,
• and the PR scope is clean and focused.

A few small improvements are recommended before merge:

1. Fix minor wording/typo issues:

• “Upload a API url” → “Upload an API URL”
• “reponse” → “response”
• “datatable” → “dataset” or “data table”

2. os module does not really fit well as a project skill/tech stack item. Consider replacing or removing it.

3. The starter code skeleton would be stronger if basic function stubs were included (for example fetch_api_data(), clean_data(), etc.) instead of comments only.

Overall, this is a solid intermediate-level project addition and a valuable contribution to the repository.

[Harshan07-web]

@komalharshita Thank you for being specific about the improvements that were required, I have taken care of :
1. Typos and Some grammatical errors in the projects.json
2. Enhanced the starter code. Now the starter code contains all core functions specifying what to be added inside them, and a main pipeline to run after the user completes the core functions.
Thanks for the feedback!

[komalharshita]

@Harshan07-web kindly resolve the merge conflicts

[Harshan07-web]

@komalharshita I have resolved the merge conflict.