Skip to content

README.md

Latest commit

 

History

History
582 lines (422 loc) · 14.5 KB

README.md

File metadata and controls

582 lines (422 loc) · 14.5 KB
Raw

RemitLend Backend API

Express.js backend service for the RemitLend platform, providing API endpoints for credit scoring, remittance simulation, and NFT metadata management.

Overview

The backend serves as a bridge between the frontend application and the Stellar blockchain, handling:

  • Loan Event Indexing: A robust polling service that watches Soroban RPC for LoanRequested, LoanApproved, and LoanRepaid events.
  • Credit Scoring: Generation and verification of borrower scores based on indexed history.
  • Remittance Simulation: API support for generating mocked remittance data for testing.
  • NFT Metadata: Serving metadata for the Remittance NFT collection.
  • Security: Request validation, rate limiting, and centralized error handling.

Tech Stack

  • Runtime: Node.js 18+
  • Framework: Express.js 5
  • Language: TypeScript
  • Validation: Zod
  • Testing: Jest + Supertest
  • Documentation: Swagger/OpenAPI
  • Code Quality: ESLint + Prettier

Getting Started

Prerequisites

  • Node.js 18 or higher
  • npm or yarn

Installation

# Install dependencies
npm install

# Copy environment variables
cp .env.example .env

# Apply database migrations (requires PostgreSQL and DATABASE_URL in .env)
npm run migrate:up

# Start development server
npm run dev

Database and migrations

The API expects a PostgreSQL database. Set DATABASE_URL in .env (see .env.example).

Apply schema migrations from the backend directory:

npm run migrate:up

Rollback last batch (when needed):

npm run migrate:down

Scripts use migrate:up and migrate:down (colon-separated names), which work reliably across shells and CI.

Core tables are created by these migrations (run in filename order):

Migration Tables
1771691269865_initial-schema.js scores, remittance_history
1771691269866_loan-events-schema.js loan_events, indexer_state
1772000000000_webhook-subscriptions.js webhook_subscriptions
1773000000001_user-profiles.js user_profiles
1773000000002_loan-history.js loan_history
1773000000003_indexed-events.js indexed_events
1774000000004_scores-add-created-at.js adds created_at to scores (idempotent)
1777000000008_unique-loan-status-events.js dedupes and enforces unique status events per loan

Migration naming convention

Each migration file follows the pattern <timestamp>_<description>.js. The timestamp prefix is a 13-digit Unix millisecond value (generated by Date.now().toString()) that determines apply order — migrations run in strictly ascending timestamp order.

To avoid collisions:

  • Always use npm run migrate:create <name> to generate new migrations — it calls Date.now() automatically and guarantees uniqueness.
  • If you must create a migration file manually, first check the largest existing timestamp in migrations/ and ensure your new prefix is strictly larger.
  • Never submit a PR where two migration files share the same numeric prefix — the apply order becomes non-deterministic.

Renaming an already-applied migration (existing databases)

If you rename a migration file that has already been applied to a database, node-pg-migrate will reject the run with:

Error: Not run migration <new-name> is preceding already run migration <old-name>

This happens because pgmigrations still references the old filename while the filesystem has the new one. Fix it by syncing the tracking table before re-running npm run migrate:up:

UPDATE pgmigrations
SET name = '1777000000008_unique-loan-status-events'
WHERE name = '1777000000007_unique-loan-status-events';

UPDATE pgmigrations
SET name = '1778000000009_transaction-submissions'
WHERE name = '1778000000008_transaction-submissions';

UPDATE pgmigrations
SET name = '1786000000017_webhook-max-attempts'
WHERE name = '1786000000016_webhook-max-attempts';

UPDATE pgmigrations
SET name = '1788000000019_unified-contract-events'
WHERE name = '1788000000018_unified-contract-events';

Afterwards, npm run migrate:up will run normally.

With Docker Compose from the repo root, the backend service runs migrate:up before npm run dev so the schema is applied automatically when the database is healthy.

Environment Variables

Create a .env file in the backend directory (see .env.example for all available variables):

# Server Configuration
PORT=3001

# CORS Configuration
FRONTEND_URL=http://localhost:3000

# Stellar Configuration
STELLAR_NETWORK=testnet
STELLAR_RPC_URL=https://soroban-testnet.stellar.org
STELLAR_NETWORK_PASSPHRASE=Test SDF Network ; September 2015
LOAN_MANAGER_CONTRACT_ID=
LOAN_MANAGER_ADMIN_SECRET=S...   # valid Stellar Ed25519 secret seed

Secret format requirements

The server validates secret format at startup and exits if any check fails:

Variable Requirement
JWT_SECRET At least 32 characters
INTERNAL_API_KEY At least 32 characters
LOAN_MANAGER_ADMIN_SECRET Valid Stellar Ed25519 secret seed (S..., 56 characters)

Available Scripts

# Development
npm run dev          # Start dev server with hot reload

# Database
npm run migrate:up   # Apply migrations (requires DATABASE_URL)
npm run migrate:down # Roll back last migration batch
npm run seed         # Seed realistic local development data
npm run seed:reset   # Reset and reseed development data

# Production
npm run build        # Compile TypeScript to JavaScript
npm start            # Run production build

# Testing
npm test             # Run test suite
npm test -- --watch  # Run tests in watch mode
npm test -- --coverage  # Run tests with coverage

# Code Quality
npm run lint         # Check code quality
npm run lint:fix     # Fix linting issues
npm run format       # Format code with Prettier
npm run format:check # Check code formatting

Development seed data

New contributors can populate a realistic local dataset after running migrations:

npm run seed

This seeds:

  • user_profiles with sample borrowers and a lender
  • scores with varied borrower scores
  • remittance_history with completed, late, missed, and pending records
  • loan_history with pending, active, repaid, and defaulted loans
  • loan_events so borrower dashboards, loan details, pool stats, and SSE endpoints have data
  • notifications with both read and unread sample messages
  • indexer_state so interest calculations have a seeded latest ledger

To wipe those local development rows and recreate them from scratch:

npm run seed:reset

API Endpoints

Health Check

GET /api/health

Check if the API is running.

Response:

{
  "status": "ok",
  "timestamp": "2024-01-15T10:30:00.000Z"
}

Credit Score

GET /api/score/:userId

Get the credit score for a specific user based on their remittance history.

Parameters:

  • userId (string) - User identifier

Response:

{
  "userId": "user123",
  "score": 750,
  "history": {
    "totalTransactions": 24,
    "averageAmount": 500,
    "consistency": 0.95
  },
  "calculatedAt": "2024-01-15T10:30:00.000Z"
}

Error Responses:

  • 400 - Invalid user ID
  • 404 - User not found
  • 500 - Server error

Simulate Remittance

POST /api/score/simulate

Simulate remittance history for testing purposes.

Request Body:

{
  "userId": "user123",
  "transactions": [
    {
      "amount": 500,
      "date": "2024-01-01",
      "recipient": "family_member"
    }
  ]
}

Response:

{
  "userId": "user123",
  "score": 750,
  "simulationId": "sim_abc123"
}

API Documentation

Interactive API documentation is available via Swagger UI when the server is running:

URL: http://localhost:3001/api-docs

The Swagger documentation provides:

  • Complete endpoint specifications
  • Request/response schemas
  • Interactive API testing
  • Authentication details (when implemented)

Project Structure

backend/
├── src/
│   ├── __tests__/           # Test files
│   │   ├── health.test.ts
│   │   ├── score.test.ts
│   │   ├── validation.test.ts
│   │   └── errorHandling.test.ts
│   ├── config/              # Configuration files
│   │   └── swagger.ts       # Swagger/OpenAPI config
│   ├── controllers/         # Request handlers
│   │   ├── scoreController.ts
│   │   └── simulationController.ts
│   ├── middleware/          # Express middleware
│   │   ├── asyncHandler.ts  # Async error wrapper
│   │   ├── auth.ts          # Authentication (planned)
│   │   ├── errorHandler.ts  # Error handling
│   │   ├── rateLimiter.ts   # Rate limiting
│   │   └── validation.ts    # Request validation
│   ├── routes/              # API routes
│   │   └── index.ts
│   ├── schemas/             # Zod validation schemas
│   │   ├── scoreSchemas.ts
│   │   └── simulationSchemas.ts
│   ├── errors/              # Custom error classes
│   │   └── AppError.ts
│   ├── app.ts               # Express app setup
│   └── index.ts             # Server entry point
├── .env.example             # Environment template
├── .eslintrc.cjs            # ESLint configuration
├── .prettierrc              # Prettier configuration
├── jest.config.js           # Jest configuration
├── tsconfig.json            # TypeScript configuration
├── package.json
└── README.md

Middleware

Error Handler

Centralized error handling middleware that catches and formats errors.

import { errorHandler } from "./middleware/errorHandler";
app.use(errorHandler);

Validation

Request validation using Zod schemas.

import { validate } from "./middleware/validation";
import { mySchema } from "./schemas/mySchemas";

router.post("/endpoint", validate(mySchema), controller);

Rate Limiter

Protects endpoints from abuse with configurable rate limits.

import { rateLimiter } from "./middleware/rateLimiter";
app.use("/api/", rateLimiter);

Async Handler

Wraps async route handlers to catch errors automatically.

import { asyncHandler } from "./middleware/asyncHandler";

router.get(
  "/endpoint",
  asyncHandler(async (req, res) => {
    // Async code here
  }),
);

Testing

Running Tests

# Run all tests
npm test

# Run specific test file
npm test -- health.test.ts

# Run with coverage
npm test -- --coverage

# Watch mode
npm test -- --watch

Test Structure

import request from "supertest";
import app from "../app";

describe("GET /api/health", () => {
  it("should return 200 OK", async () => {
    const response = await request(app).get("/api/health").expect(200);

    expect(response.body).toHaveProperty("status", "ok");
  });
});

Test Coverage

Aim for >80% code coverage on new code. Current coverage:

  • Statements: Check with npm test -- --coverage
  • Branches: Check with npm test -- --coverage
  • Functions: Check with npm test -- --coverage
  • Lines: Check with npm test -- --coverage

Error Handling

Custom Error Class

import { AppError } from "./errors/AppError";

throw new AppError("User not found", 404);

Error Response Format

{
  "success": false,
  "message": "Error message",
  "statusCode": 400,
  "errors": []
}

Validation Schemas

Validation schemas are defined using Zod in the schemas/ directory.

Example Schema

import { z } from "zod";

export const getUserScoreSchema = z.object({
  params: z.object({
    userId: z.string().min(1, "User ID is required"),
  }),
});

export const simulateRemittanceSchema = z.object({
  body: z.object({
    userId: z.string().min(1),
    transactions: z.array(
      z.object({
        amount: z.number().positive(),
        date: z.string(),
        recipient: z.string(),
      }),
    ),
  }),
});

Future Enhancements

Phase 1: Real Remittance Integration

  • Wise API integration
  • Western Union API integration
  • Remittance data verification
  • Historical data import

Phase 2: Enhanced Features

  • User authentication (JWT)
  • Database integration (PostgreSQL)
  • IPFS integration for NFT metadata
  • Webhook support for blockchain events

Phase 3: Production Ready

  • Caching layer (Redis)
  • Logging and monitoring
  • CI/CD pipeline
  • Load balancing support
  • API versioning

Deployment

Docker

# Build image
docker build -t remitlend-backend .

# Run container
docker run -p 3001:3001 --env-file .env remitlend-backend

Docker Compose

From the project root:

docker compose up backend

Production Considerations

  • Set NODE_ENV=production
  • Use process manager (PM2, systemd)
  • Enable HTTPS
  • Configure proper CORS origins
  • Set up monitoring and logging
  • Implement health checks
  • Use environment-specific configs

Contributing

See CONTRIBUTING.md for guidelines on contributing to the backend.

Code Style

  • Follow TypeScript best practices
  • Use async/await over callbacks
  • Maintain strict typing
  • Write descriptive variable names
  • Add JSDoc comments for public functions
  • Keep functions small and focused

Before Submitting PR

# Run all checks
npm run lint
npm run format:check
npm test
npm run build

Troubleshooting

Port Already in Use

# Find and kill process on port 3001
lsof -ti:3001 | xargs kill -9

TypeScript Errors

# Clean build
rm -rf dist/
npm run build

Module Not Found

# Reinstall dependencies
rm -rf node_modules package-lock.json
npm install

License

ISC License - See LICENSE file for details.

Support

  • Open an issue for bug reports
  • Check existing issues before creating new ones
  • Provide detailed reproduction steps
  • Include error messages and logs