Developing a Task Management Microservice with Node.js

In this article, I will demonstrate how to build a microservice using Node.js, using a Task Management web service as an example. This microservice will provide a simple yet powerful API to create, retrieve, and update tasks. We will focus on making the microservice scalable, robust, reliable, and performant using popular Node.js stack components.

API Overview

Our Task Management API will support the following operations:

1. Create a Task: Allows creating a task with a name and description, with the status set to ‘new’.
2. Get a Task: Retrieves a task by its identifier.
3. Update a Task: Updates the task’s status, name, and description.

Application Requirements

• Task Creation: New tasks should be created with the status ‘new’.
• Status Transitions:
• ‘new’ → ‘active’
• ‘new’ → ‘canceled’
• ‘active’ → ‘completed’
• ‘active’ → ‘canceled’
• Race Condition Avoidance: Ensure consistent updates without race conditions.

Non-functional Requirements

• Scalability: Handle increasing requests efficiently.
• Elasticity: Manage spikes in traffic smoothly.
• Performance: Provide quick responses for better user experience.
• Resilience: Be fault-tolerant and capable of recovering gracefully.
• Monitoring & Observability: Track health, logs, and metrics.
• Testability: Ensure the service is easy to test.
• Statelessness: Store client context in the database, not in the service.
• Deployability: Easy to deploy and update.

Technology Stack

Node.js

Node.js is chosen for its speed, large community, and suitability for both frontend and backend development.

Database

MongoDB: A document-oriented NoSQL database that offers:

• Schemaless Storage: Collections can hold documents with varying schemas.
• Scalability: Designed for horizontal scaling.
• Performance: Optimized for read-heavy workloads.

Web Framework

Express: A minimal and flexible Node.js web application framework.

Validation

Joi: A powerful schema description and data validation library. Express-mongo-sanitize: Middleware to prevent MongoDB operator injection.

Configuration

Dotenv: A module to load environment variables from a .env file.

Static Analysis

ESLint: A tool for identifying and reporting on patterns found in ECMAScript/JavaScript code, with a focus on code quality.

Testing

Jest: A JavaScript testing framework to ensure the correctness of code through unit and integration tests.

Logging

Winston: A versatile logging library with support for multiple transports.

Metrics

Prometheus Middleware: For collecting standard web application metrics.

Monitoring Stack

• Prometheus: Monitoring and alerting toolkit.
• Promtail: A log collector and shipper.
• Loki: Log aggregation system.
• Grafana: A visualization and analytics platform.

Local Infrastructure

Docker: To create a local environment similar to production. Docker Compose: To define and manage multi-container Docker applications.

Continuous Integration

GitHub Actions: For CI/CD, ensuring new commits do not break the build.

Application Development

We’ll create a Node.js application using the stack above, with a MongoDB backend.

Project Structure

Here is an example of the directory structure:

.
├── src
│   ├── config
│   │   ├── config.js
│   │   ├── logger.js
│   ├── controllers
│   │   └── task.js
│   ├── middlewares
│   │   ├── validate.js
│   │   └── error.js
│   ├── models
│   │   └── task.js
│   ├── routes
│   │   ├── v1
│   │   │   └── task.js
│   ├── services
│   │   └── task.js
│   ├── utils
│   │   └── pick.js
│   ├── validations
│   │   └── task.js
│   ├── app.js
│   ├── index.js
├── tests
│   ├── integration
│   │   └── task.test.js
│   ├── unit
│   │   └── task.test.js
├── .env
├── .eslintrc.json
├── docker-compose.yml
├── Dockerfile
├── jest.config.js
└── package.json

Sample Code Snippets

Task Model

Define the Task schema using Mongoose:

const mongoose = require('mongoose');
const { Schema } = mongoose;

const TaskSchema = new Schema(
  {
    name: {
      type: String,
      required: true,
    },
    description: {
      type: String,
      required: false,
    },
    status: {
      type: String,
      enum: ['new', 'active', 'completed', 'cancelled'],
      default: 'new',
    },
    createdAt: {
      type: Date,
      default: Date.now,
    },
    updatedAt: Date,
  },
  { optimisticConcurrency: true }
);

module.exports = mongoose.model('task', TaskSchema);

Task Controller

Implement the controller to handle task updates:

const updateTaskById = catchAsync(async (req, res) => {
  const result = await taskService.updateTaskById(req.params.id, req.body);
  if (result.error) {
    switch (result.code) {
      case taskService.errorCodes.AT_LEAST_ONE_UPDATE_REQUIRED_CODE:
        res.status(400).json({ success: false, message: 'at least one update required' });
        return;
      case taskService.errorCodes.INVALID_STATUS_CODE:
        res.status(400).json({ success: false, message: 'invalid status' });
        return;
      case taskService.errorCodes.INVALID_STATUS_TRANSITION_CODE:
        res.status(404).json({ success: false, message: 'task not found' });
        return;
      case taskService.errorCodes.TASK_NOT_FOUND_CODE:
        res.status(400).json({ success: false, message: result.error });
        return;
      case taskService.errorCodes.CONCURRENCY_ERROR_CODE:
        res.status(500).json({ success: false, message: 'concurrency error' });
        return;
      default:
        res.status(500).json({ success: false, message: 'internal server error' });
        return;
    }
  }

  res.status(200).json({
    success: true,
    task: toDto(result),
  });
});

Task Service

Implement the service logic to handle business rules and data persistence:

async function updateTaskById(id, { name, description, status }) {
  if (!name && !description && !status) {
    return { error: 'at least one update required', code: AT_LEAST_ONE_UPDATE_REQUIRED_CODE };
  }

  if (status && !(status in availableUpdates)) {
    return { error: 'invalid status', code: INVALID_STATUS_CODE };
  }
  
  for (let retry = 0; retry < 3; retry += 1) {
    const task = await Task.findById(id);
    if (!task) {
      return { error: 'task not found', code: TASK_NOT_FOUND_CODE };
    }
    if (status) {
      const allowedStatuses = availableUpdates[task.status];
      if (!allowedStatuses.includes(status)) {
        return {
          error: `cannot update from '${task.status}' to '${status}'`,
          code: INVALID_STATUS_TRANSITION_CODE,
        };
      }
    }
    task.status = status ?? task.status;
    task.name = name ?? task.name;
    task.description = description ?? task.description;
    task.updatedAt = Date.now();
    try {
      await task.save();
    } catch (error) {
      if (error.name === 'VersionError') {
        continue;
      }
    }
    return task;
  }
  return { error: 'concurrency error', code: CONCURRENCY_ERROR_CODE };
}

Routes

Register the routes and apply validation middleware:

const { Router } = require('express');
const taskController = require('../../../controllers/task');
const taskValidation = require('../../../validation/task');
const validate = require('../../../middlewares/validate');

const router = Router();
router.get('/:id', validate(taskValidation.getTaskById), taskController.getTaskById);
router.put('/', validate(taskValidation.createTask), taskController.createTask);
router.post('/:id', validate(taskValidation.updateTaskById), taskController.updateTaskById);

module.exports = router;

Testing

Implement integration tests to verify the API functionality:

describe('Task API', () => {
  setupServer();

  it('should create and update a task', async () => {
    let response = await fetch('/v1/tasks', {
      method: 'put',
      body: JSON.stringify({
        name: 'Test Task',
        description: 'Task description',
      }),
      headers: { 'Content-Type': 'application/json' },
    });
    expect(response.status).toEqual(201);
    const result = await response.json();
    expect(result.success).toBe(true);
    const taskId = result.task.id;

    response = await fetch(`/v1/tasks/${taskId}`, {
      method: 'post',
      body: JSON.stringify({ status: 'active' }),
      headers: { 'Content-Type': 'application/json' },
    });
    expect(response.status).toEqual(200);
    const updateResult = await response.json();
    expect(updateResult.success).toBe(true);
    expect(updateResult.task.status).toBe('active');
  });
});

Deployment

Deploy the microservice using Docker and Docker Compose. Here’s an example docker-compose.yml:

version: '3.8'
services:
  mongo:
    image: mongo
    container_name: mongodb
    ports:
      - '27017:27017'
    volumes:
      - mongo-data:/data/db

  api:
    build: .
    container_name: task_api
    ports:
      - '3000:3000'
    depends_on:
      - mongo
    environment:
      - MONGO_URL=mongodb://mongo:27017/taskdb
    volumes:
      - .:/usr/src/app

volumes:
  mongo-data:

Conclusion

By following this guide, you can create a robust, scalable, and reliable microservice for managing tasks. This architecture ensures the microservice can handle concurrent updates without issues, supports smooth deployment, and offers easy scalability and monitoring.