Skip to content

Commit 07f953a

Browse files
Sakeeb91Sakeeb91
committed
feat: implement standardized health check system (issue #5)
- Create @scribemed/health package with reusable health check utilities - Add /health, /health/live, and /health/ready endpoints to all services - Implement database connectivity checks for documentation service - Add Kubernetes liveness and readiness probes to service deployments - Add comprehensive unit and integration tests - Update documentation with health check usage examples Closes #5
1 parent 8f6a790 commit 07f953a

18 files changed

Lines changed: 1139 additions & 14 deletions

File tree

‎docs/issues/0005-health-check-system.md‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Issue #5: Implement Standardized Health Check System
2+
3+
## Summary
4+
5+
Implement a standardized health check system across all microservices to enable reliable monitoring, load balancing, and orchestration. This system should provide consistent health endpoints that can be consumed by Kubernetes liveness/readiness probes, API gateways, and monitoring systems.
6+
7+
## Background
8+
9+
Currently, services lack standardized health check endpoints. This makes it difficult to:
10+
11+
- Configure Kubernetes liveness and readiness probes
12+
- Monitor service health in production
13+
- Implement proper load balancing
14+
- Detect and handle degraded service states
15+
- Integrate with monitoring and alerting systems
16+
17+
## Proposed Changes
18+
19+
1. **Create shared health check package**
20+
- Add `packages/health/` package with reusable health check utilities
21+
- Support for basic health, readiness, and liveness checks
22+
- Database connectivity checks
23+
- Dependency health checks (external services, etc.)
24+
25+
2. **Implement health endpoints in all services**
26+
- Add `/health`, `/health/ready`, and `/health/live` endpoints
27+
- Integrate with existing services (transcription, documentation, coding)
28+
- Return standardized JSON responses
29+
30+
3. **Update Kubernetes configurations**
31+
- Add liveness and readiness probes to service deployments
32+
- Configure appropriate timeouts and thresholds
33+
34+
4. **Add health check tests**
35+
- Unit tests for health check logic
36+
- Integration tests for health endpoints
37+
38+
## Acceptance Criteria
39+
40+
- [x] `packages/health/` package created with reusable health check utilities
41+
- [x] All services expose `/health`, `/health/ready`, and `/health/live` endpoints
42+
- [x] Health endpoints return standardized JSON responses
43+
- [x] Database connectivity is checked in readiness probes
44+
- [x] Kubernetes manifests updated with liveness/readiness probes
45+
- [x] Unit and integration tests added for health checks
46+
- [x] Documentation updated with health check usage
47+
48+
## Implementation Details
49+
50+
### Health Check Types
51+
52+
- **Liveness**: Indicates if the service is running (should always return 200 if process is alive)
53+
- **Readiness**: Indicates if the service is ready to accept traffic (checks dependencies like DB)
54+
- **Health**: Comprehensive health status including dependencies
55+
56+
### Response Format
57+
58+
```json
59+
{
60+
"status": "healthy" | "degraded" | "unhealthy",
61+
"timestamp": "2024-01-01T00:00:00Z",
62+
"checks": {
63+
"database": {
64+
"status": "healthy",
65+
"responseTime": 5
66+
},
67+
"memory": {
68+
"status": "healthy",
69+
"usage": 45.2
70+
}
71+
}
72+
}
73+
```
74+
75+
## Status: Completed
76+
77+
## Related Issues
78+
79+
- Issue #2: Monorepo Developer Experience
80+
- Issue #15: CI/CD Pipeline (health checks needed for deployment)

‎infrastructure/kubernetes/staging/coding-service.yaml‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: coding-service
5+
namespace: staging
6+
labels:
7+
app: coding-service
8+
app.kubernetes.io/name: scribemed
9+
app.kubernetes.io/environment: staging
10+
spec:
11+
replicas: 2
12+
selector:
13+
matchLabels:
14+
app: coding-service
15+
template:
16+
metadata:
17+
labels:
18+
app: coding-service
19+
app.kubernetes.io/name: scribemed
20+
app.kubernetes.io/environment: staging
21+
spec:
22+
containers:
23+
- name: coding-service
24+
image: scribemed/coding-service:latest
25+
ports:
26+
- containerPort: 8082
27+
name: http
28+
env:
29+
- name: PORT
30+
value: '8082'
31+
- name: ENVIRONMENT
32+
value: staging
33+
livenessProbe:
34+
httpGet:
35+
path: /health/live
36+
port: http
37+
initialDelaySeconds: 10
38+
periodSeconds: 10
39+
timeoutSeconds: 5
40+
failureThreshold: 3
41+
readinessProbe:
42+
httpGet:
43+
path: /health/ready
44+
port: http
45+
initialDelaySeconds: 5
46+
periodSeconds: 5
47+
timeoutSeconds: 3
48+
failureThreshold: 3
49+
resources:
50+
requests:
51+
memory: '256Mi'
52+
cpu: '100m'
53+
limits:
54+
memory: '512Mi'
55+
cpu: '500m'
56+
---
57+
apiVersion: v1
58+
kind: Service
59+
metadata:
60+
name: coding-service
61+
namespace: staging
62+
labels:
63+
app: coding-service
64+
app.kubernetes.io/name: scribemed
65+
spec:
66+
type: ClusterIP
67+
ports:
68+
- port: 80
69+
targetPort: http
70+
protocol: TCP
71+
name: http
72+
selector:
73+
app: coding-service
74+

‎infrastructure/kubernetes/staging/documentation-service.yaml‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: documentation-service
5+
namespace: staging
6+
labels:
7+
app: documentation-service
8+
app.kubernetes.io/name: scribemed
9+
app.kubernetes.io/environment: staging
10+
spec:
11+
replicas: 2
12+
selector:
13+
matchLabels:
14+
app: documentation-service
15+
template:
16+
metadata:
17+
labels:
18+
app: documentation-service
19+
app.kubernetes.io/name: scribemed
20+
app.kubernetes.io/environment: staging
21+
spec:
22+
containers:
23+
- name: documentation-service
24+
image: scribemed/documentation-service:latest
25+
ports:
26+
- containerPort: 8081
27+
name: http
28+
env:
29+
- name: PORT
30+
value: '8081'
31+
- name: ENVIRONMENT
32+
value: staging
33+
- name: DB_HOST
34+
valueFrom:
35+
secretKeyRef:
36+
name: database-credentials
37+
key: host
38+
- name: DB_PORT
39+
value: '5432'
40+
- name: DB_NAME
41+
valueFrom:
42+
secretKeyRef:
43+
name: database-credentials
44+
key: database
45+
- name: DB_USER
46+
valueFrom:
47+
secretKeyRef:
48+
name: database-credentials
49+
key: user
50+
livenessProbe:
51+
httpGet:
52+
path: /health/live
53+
port: http
54+
initialDelaySeconds: 10
55+
periodSeconds: 10
56+
timeoutSeconds: 5
57+
failureThreshold: 3
58+
readinessProbe:
59+
httpGet:
60+
path: /health/ready
61+
port: http
62+
initialDelaySeconds: 10
63+
periodSeconds: 5
64+
timeoutSeconds: 3
65+
failureThreshold: 3
66+
resources:
67+
requests:
68+
memory: '256Mi'
69+
cpu: '100m'
70+
limits:
71+
memory: '512Mi'
72+
cpu: '500m'
73+
---
74+
apiVersion: v1
75+
kind: Service
76+
metadata:
77+
name: documentation-service
78+
namespace: staging
79+
labels:
80+
app: documentation-service
81+
app.kubernetes.io/name: scribemed
82+
spec:
83+
type: ClusterIP
84+
ports:
85+
- port: 80
86+
targetPort: http
87+
protocol: TCP
88+
name: http
89+
selector:
90+
app: documentation-service
91+

‎infrastructure/kubernetes/staging/transcription-service.yaml‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: transcription-service
5+
namespace: staging
6+
labels:
7+
app: transcription-service
8+
app.kubernetes.io/name: scribemed
9+
app.kubernetes.io/environment: staging
10+
spec:
11+
replicas: 2
12+
selector:
13+
matchLabels:
14+
app: transcription-service
15+
template:
16+
metadata:
17+
labels:
18+
app: transcription-service
19+
app.kubernetes.io/name: scribemed
20+
app.kubernetes.io/environment: staging
21+
spec:
22+
containers:
23+
- name: transcription-service
24+
image: scribemed/transcription-service:latest
25+
ports:
26+
- containerPort: 8080
27+
name: http
28+
env:
29+
- name: PORT
30+
value: '8080'
31+
- name: ENVIRONMENT
32+
value: staging
33+
livenessProbe:
34+
httpGet:
35+
path: /health/live
36+
port: http
37+
initialDelaySeconds: 10
38+
periodSeconds: 10
39+
timeoutSeconds: 5
40+
failureThreshold: 3
41+
readinessProbe:
42+
httpGet:
43+
path: /health/ready
44+
port: http
45+
initialDelaySeconds: 5
46+
periodSeconds: 5
47+
timeoutSeconds: 3
48+
failureThreshold: 3
49+
resources:
50+
requests:
51+
memory: '256Mi'
52+
cpu: '100m'
53+
limits:
54+
memory: '512Mi'
55+
cpu: '500m'
56+
---
57+
apiVersion: v1
58+
kind: Service
59+
metadata:
60+
name: transcription-service
61+
namespace: staging
62+
labels:
63+
app: transcription-service
64+
app.kubernetes.io/name: scribemed
65+
spec:
66+
type: ClusterIP
67+
ports:
68+
- port: 80
69+
targetPort: http
70+
protocol: TCP
71+
name: http
72+
selector:
73+
app: transcription-service
74+

0 commit comments

Comments
 (0)