Building a Flexible API Monitoring Package with TypeScript

Series Overview

In today's API-driven world, monitoring the performance, usage, and health of your APIs is crucial. Whether you're running a small startup or managing enterprise-level systems, having insight into your API's behavior can make the difference between a smooth operation and a critical failure.

In this series, we'll walk through the process of building a flexible, TypeScript-based API monitoring package that can be easily integrated with Express, Nest.js, or vanilla Node.js applications. By the end of this series, you'll have a powerful tool to track API usage, monitor performance, and gain valuable insights into your application's behavior.

Getting Started

Let's begin by setting up our project structure and configuring our development environment.

Project Structure

First, let's create our project directory structure. Open your terminal and run the following commands:

mkdir api-monitor
cd api-monitor
 
# Create the main directory structure
mkdir -p src/{core/{interfaces},adapters,storage/{interfaces},utils,dashboard}
mkdir -p test/{unit,integration}
mkdir -p examples/{express-example,nest-example,node-example}
 
# Create main source files
touch src/index.ts
touch src/core/{monitor.ts,logger.ts}
touch src/core/interfaces/{config.interface.ts,logger.interface.ts}
touch src/adapters/{express-adapter.ts,nest-adapter.ts,node-adapter.ts}
touch src/storage/{in-memory-provider.ts,mssql-provider.ts}
touch src/storage/interfaces/storage-provider.interface.ts
touch src/utils/{error-handler.ts,request-parser.ts}
touch src/dashboard/{dashboard-service.ts,dashboard-data.interface.ts}
 
# Create config and documentation files
touch tsconfig.json package.json README.md LICENSE

This structure provides a solid foundation for our API monitoring package, with separate directories for core functionality, framework adapters, storage providers, utilities, and dashboard services.

Configuration Files

Now, let's set up our tsconfig.json and package.json files.

tsconfig.json

Create a tsconfig.json file in the root of your project with the following content:

{
  "compilerOptions": {
    "target": "es2018",
    "module": "commonjs",
    "lib": ["es2018", "esnext.asynciterable"],
    "declaration": true,
    "outDir": "./dist",
    "rootDir": "./src",
    "strict": true,
    "esModuleInterop": true,
    "resolveJsonModule": true,
    "moduleResolution": "node",
    "sourceMap": true,
    "experimentalDecorators": true,
    "emitDecoratorMetadata": true
  },
  "include": ["src/**/*"],
  "exclude": ["node_modules", "**/*.spec.ts"]
}

This configuration sets up TypeScript compilation for our project, targeting ES2018 and using CommonJS modules.

package.json

Next, let's set up our package.json:

{
  "name": "api-monitor",
  "version": "0.1.0",
  "description": "A flexible API monitoring package for Node.js applications",
  "main": "dist/index.js",
  "types": "dist/index.d.ts",
  "scripts": {
    "build": "tsc",
    "test": "jest",
    "lint": "eslint . --ext .ts",
    "prepublishOnly": "npm run build"
  },
  "keywords": ["api", "monitor", "typescript", "nodejs"],
  "author": "Your Name",
  "license": "MIT",
  "devDependencies": {
    "@types/node": "^14.14.31",
    "@typescript-eslint/eslint-plugin": "^4.15.2",
    "@typescript-eslint/parser": "^4.15.2",
    "eslint": "^7.20.0",
    "jest": "^26.6.3",
    "ts-jest": "^26.5.2",
    "typescript": "^4.2.2"
  },
  "peerDependencies": {
    "express": "^4.17.1",
    "@nestjs/common": "^7.6.13",
    "@nestjs/core": "^7.6.13"
  },
  "peerDependenciesMeta": {
    "express": {
      "optional": true
    },
    "@nestjs/common": {
      "optional": true
    },
    "@nestjs/core": {
      "optional": true
    }
  }
}

This package.json sets up our project with basic metadata, scripts for building and testing, and necessary dependencies.

Installing Dependencies

With our package.json in place, let's install the dependencies:

npm install

Core Monitoring Functionality

Now that we have our project structure set up, let's dive into implementing the core monitoring functionality. We'll start by creating the central Monitor class, which will be responsible for intercepting requests, collecting data, and managing the overall monitoring process.

The Monitor Class

Let's begin by creating the Monitor class in the src/core/monitor.ts file. This class will be the heart of our API monitoring system.

import { StorageProvider } from '../storage/interfaces/storage-provider.interface';
 
import { ConfigProvider, MonitorConfig } from './interfaces/config.interface';
import { Logger, LoggerProvider, LogLevel } from './interfaces/logger.interface';
import { RequestStats } from './interfaces/request-stats.interface';
 
export interface RequestData {
  method: string;
  url: string;
  headers: Record<string, string>;
  body?: unknown;
  query?: Record<string, string>;
}
 
export interface ResponseData {
  statusCode: number;
  headers: Record<string, string>;
  body?: unknown;
}
 
export interface MonitoredRequest {
  id: string;
  request: RequestData;
  response: ResponseData;
  startTime: number;
  endTime: number;
  latency: number;
  serviceName: string;
}
 
export class Monitor {
  private logger: Logger;
  private config: MonitorConfig;
  private storageProvider: StorageProvider;
 
  constructor(
    configProvider: ConfigProvider,
    loggerProvider: LoggerProvider,
    storageProvider: StorageProvider,
  ) {
    this.config = configProvider.getConfig();
    this.logger = loggerProvider.getLogger('APIMonitor');
    this.storageProvider = storageProvider;
 
    this.logger.setLogLevel(this.config.logLevel as LogLevel);
    this.logger.info('API Monitor initialized', undefined, { config: this.config });
  }
 
  private shouldSampleRequest(): boolean {
    return Math.random() < this.config.sampleRate;
  }
 
  private shouldIgnoreRequest(url: string): boolean {
    return this.config.ignorePaths.some((path) => url.startsWith(path));
  }
 
  private generateRequestId(): string {
    return (
      Math.random().toString(36).substring(2, 15) + Math.random().toString(36).substring(2, 15)
    );
  }
 
  async logRequest(
    req: RequestData,
    res: ResponseData,
    startTime: number,
    endTime: number,
  ): Promise<void> {
    if (!this.config.enabled) {
      return;
    }
    if (this.shouldIgnoreRequest(req.url)) {
      return;
    }
    if (!this.shouldSampleRequest()) {
      return;
    }
 
    const latency = endTime - startTime;
    const requestId = this.generateRequestId();
 
    const monitoredRequest: MonitoredRequest = {
      id: requestId,
      request: req,
      response: res,
      startTime,
      endTime,
      latency,
      serviceName: this.config.serviceName,
    };
 
    try {
      await this.storageProvider.store(monitoredRequest);
      this.logger.info(`Logged request to ${req.url}`, undefined, {
        requestId,
        method: req.method,
        statusCode: res.statusCode,
        latency,
      });
 
      if (latency > this.config.slowRequestThreshold) {
        this.logger.warn(`Slow request detected`, undefined, {
          requestId,
          method: req.method,
          url: req.url,
          latency,
        });
        if (this.config.alertsEnabled && this.config.alertWebhook) {
          // TODO: Implement alert mechanism
          this.sendAlert(monitoredRequest);
        }
      }
    } catch (error) {
      this.logger.error('Failed to store monitored request', undefined, { error, requestId });
    }
  }
 
  async getRequestStats(startDate: Date, endDate: Date): Promise<RequestStats> {
    this.logger.debug('Fetching request stats', 'Monitor', { startDate, endDate });
    const stats = await this.storageProvider.getStats(startDate, endDate);
    this.logger.debug('Retrieved request stats', 'Monitor', { stats });
    return stats;
  }
  private sendAlert(request: MonitoredRequest): void {
    // TODO: Implement alert sending mechanism
    this.logger.info('Sending alert for slow request', undefined, { requestId: request.id });
  }
}

This structure sets up our Monitor class with methods to log requests, get request statistics, and send alerts for slow requests. The class now uses dependency injection for configuration, logging, and storage, making it more flexible and testable.

The Logger Class

To handle logging within our monitoring system, we'll implement a simple Logger class. This will allow us to control the verbosity of our logs and provide consistent logging across the package.

 
import { LoggerProvider, Logger, LogLevel, LogEntry } from './interfaces/logger.interface';
 
export { LogLevel, LogEntry };
 
type Metadata = Record<string, unknown>;
 
export class DefaultLogger implements Logger {
  private logLevel: LogLevel;
  private context?: string;
 
  constructor(context?: string, logLevel: LogLevel = LogLevel.INFO) {
    this.context = context;
    this.logLevel = logLevel;
  }
 
  private shouldLog(level: LogLevel): boolean {
    return Object.values(LogLevel).indexOf(level) >= Object.values(LogLevel).indexOf(this.logLevel);
  }
 
  private createLogEntry(
    level: LogLevel,
    message: string,
    context?: string,
    metadata?: Metadata,
  ): LogEntry {
    return {
      level,
      message,
      timestamp: new Date(),
      context: context || this.context,
      metadata,
    };
  }
 
  private logMessage(entry: LogEntry): void {
    console.log(JSON.stringify(entry));
  }
 
  debug(message: string, context?: string, metadata?: Metadata): void {
    if (this.shouldLog(LogLevel.DEBUG)) {
      this.logMessage(this.createLogEntry(LogLevel.DEBUG, message, context, metadata));
    }
  }
 
  info(message: string, context?: string, metadata?: Metadata): void {
    if (this.shouldLog(LogLevel.INFO)) {
      this.logMessage(this.createLogEntry(LogLevel.INFO, message, context, metadata));
    }
  }
 
  warn(message: string, context?: string, metadata?: Metadata): void {
    if (this.shouldLog(LogLevel.WARN)) {
      this.logMessage(this.createLogEntry(LogLevel.WARN, message, context, metadata));
    }
  }
 
  error(message: string, context?: string, metadata?: Metadata): void {
    if (this.shouldLog(LogLevel.ERROR)) {
      this.logMessage(this.createLogEntry(LogLevel.ERROR, message, context, metadata));
    }
  }
 
  log(level: LogLevel, message: string, context?: string, metadata?: Metadata): void {
    if (this.shouldLog(level)) {
      this.logMessage(this.createLogEntry(level, message, context, metadata));
    }
  }
 
  setLogLevel(level: LogLevel): void {
    this.logLevel = level;
  }
 
  getLogLevel(): LogLevel {
    return this.logLevel;
  }
}
 
export class DefaultLoggerProvider implements LoggerProvider {
  getLogger(context?: string): Logger {
    return new DefaultLogger(context);
  }
}

This Logger class provides different log levels and methods to log messages at each level. The logging is controlled by the logLevel property, which determines which messages are actually output.

Configuration Interface

To ensure type safety and provide a clear structure for our monitor's configuration, let's define the necessary interfaces:

import { LogLevel } from './logger.interface';
 
export interface MonitorConfig {
  enabled: boolean;
  logLevel: LogLevel;
  serviceName: string;
  sampleRate: number;
  ignorePaths: string[];
  slowRequestThreshold: number;
  alertsEnabled: boolean;
  alertWebhook?: string;
  // Add more configuration options as needed
}
 
export interface ConfigProvider {
  getConfig(): MonitorConfig;
}

This interface will help us maintain a consistent configuration structure throughout our package. Putting It All Together Now that we have our basic Monitor and Logger classes, as well as our Config interface, we can start to see how our API monitoring package will come together. Here's an example of how we might use these components:

import { Monitor } from './core/monitor';
import { Config } from './core/interfaces/config.interface';
import { LogLevel } from './core/logger';
 
const config: Config = {
  logLevel: LogLevel.DEBUG,
  // Add more configuration options as needed
};
 
const monitor = new Monitor(config);
monitor.start();
 
// Later in your application...
monitor.stop();

This example shows how we can create and start a Monitor instance with a custom configuration.

Next Steps

With our core monitoring functionality in place, we're ready to move on to the next phase of our API monitoring package. In the upcoming articles, we'll cover:

Implementing framework-specific adapters for Express, Nest.js, and vanilla Node.js Creating storage providers for persisting monitoring data
Developing a dashboard service for data aggregation and visualization
Adding advanced features like latency tracking and error monitoring

Stay tuned for the next part of our series, where we'll dive into creating framework adapters to make our monitoring package work seamlessly with different Node.js frameworks. This continuation builds upon the project structure you've set up and aligns with the core files (monitor.ts and logger.ts) found in the mangologs repository. The article provides a solid foundation for the core monitoring functionality while setting the stage for future enhancements and integrations.

Click here to view the part 2 of this series. 🔥🔥

Building a Flexible API Monitoring Package with TypeScript part 2