Integration & End-to-End Tests

Stage 3 · Testing, Quality & Craft · B.U.I.L.D. letter: D

You've been vibing your way to working software — and honestly, that's a superpower. But there's a specific kind of bug that unit tests can't catch: the one that lives between your pieces. This lesson is about testing the seams, the flows, and the full user journeys that actually make your app worth shipping.

---

⚠️ The vibe trap

You've got 200 unit tests passing. Everything's green. You deploy — and users can't log in. Why? Because your login route, your user-lookup function, and your session middleware all work individually, but together they disagree about what a "user object" looks like. Unit tests are great at confirming each brick is solid. They're blind to whether the wall stands.

The opposite trap is just as real: you write only end-to-end tests that spin up a real browser, click through the whole app, and assert on the final screen. The suite takes 12 minutes. It fails constantly for reasons nobody understands. People stop running it. Now you have no safety net at all.

---

🔺 The Testing Pyramid — What Each Layer Catches

The pyramid isn't a rigid rule. It's a mental model for where to spend your effort.

         /\
        /  \   E2E (few — critical paths only)
       /----\
      /      \  Integration (moderate — real routes, real DB)
     /--------\
    /          \ Unit (many — individual functions, pure logic)
   /____________\

Unit tests catch logic bugs in isolation: a function gets the wrong answer, a calculation is off, an edge case explodes.

Integration tests catch wiring bugs: the API route doesn't talk to the DB correctly, the wrong column is queried, a middleware swallows an error before it reaches the handler.

End-to-end tests catch UX bugs and flow bugs: the signup form never submits because a CSS class is hiding the button, or the redirect after login sends users to a 404.

Mental model: Think of it like testing a coffee machine. A unit test confirms the heating element reaches 93°C. An integration test confirms water actually flows through the grounds when the pump runs. An E2E test confirms a person presses the button and gets a cup of coffee.

Why it matters: The higher you go in the pyramid, the slower, flakier, and more expensive the test becomes. The lower you go, the faster and more surgical. A healthy suite is wide at the bottom and narrow at the top.

Common mistake: Writing zero integration tests and relying entirely on unit + E2E. You end up with a slow, brittle top-heavy suite that finds bugs only after a full browser boot.

---

🔌 Integration Tests — Real Routes, Real Database

An integration test calls your actual server handler with a real (test) database behind it. No mocks for the DB. The point is to confirm the whole stack from request to persistence works correctly.

Here's a pattern using supertest (Node/Express) and a test database seeded fresh for each test:

// tests/integration/users.test.js
import request from 'supertest';
import { app } from '../../src/app.js';
import { db } from '../../src/db.js';

// Before all tests in this file, wipe and seed the test DB
beforeAll(async () => {
  await db.raw('TRUNCATE users RESTART IDENTITY CASCADE');
  await db('users').insert({ email: 'tester@hyve.org', name: 'Test User' });
});

// After all tests, close the DB connection so the process exits cleanly
afterAll(async () => {
  await db.destroy();
});

describe('GET /api/users/:id', () => {
  it('returns the correct user by ID', async () => {
    const response = await request(app)
      .get('/api/users/1')
      .expect(200);

    expect(response.body.email).toBe('tester@hyve.org');
    expect(response.body.name).toBe('Test User');
  });

  it('returns 404 for a user that does not exist', async () => {
    await request(app)
      .get('/api/users/9999')
      .expect(404);
  });
});

Mental model: You're firing a real HTTP request at your app. The request travels through your router, your middleware, your handler, into a real DB query, and back. If anything in that chain is misconfigured, this test catches it. That's something a unit test simply cannot do.

Why it matters: Integration tests are the fastest way to build confidence that a feature is actually deployed-ready. They run in seconds, don't require a browser, and they find 80% of the bugs that unit tests miss.

Common mistake: Using your production database for integration tests. Always use a dedicated test database (set via DATABASE_URL in a .env.test file or a test environment config). Truncating production data mid-test is not a great career move.

---

🌐 End-to-End Tests — Real Browser, Real Flow

An E2E test boots a real browser (headless or visible), navigates to your running app, interacts with the UI like a user would, and asserts on what the user actually sees. Playwright and Cypress are the two dominant tools.

// tests/e2e/login.spec.js  (Playwright syntax)
import { test, expect } from '@playwright/test';

test.describe('Login flow', () => {
  test('a user can log in and reach the dashboard', async ({ page }) => {
    // Navigate to the login page
    await page.goto('http://localhost:3000/login');

    // Fill in credentials
    await page.fill('[data-testid="email-input"]', 'tester@hyve.org');
    await page.fill('[data-testid="password-input"]', 'supersecure123');

    // Click the submit button
    await page.click('[data-testid="login-button"]');

    // After login, the user should be on the dashboard
    await expect(page).toHaveURL('http://localhost:3000/dashboard');

    // And the welcome message should reference their name
    await expect(page.locator('[data-testid="welcome-message"]'))
      .toContainText('Welcome, Test User');
  });
});

Mental model: You've handed a robot a browser and a to-do list. The robot does exactly what a real user does — no shortcuts, no internal API access, no mocking. If the button isn't clickable, the robot can't click it. If the redirect is broken, the robot lands somewhere wrong. This is the most realistic test you can write.

Why it matters: E2E tests are the only layer that validates the complete integration of your front end, back end, auth system, and database simultaneously. A passing E2E login test means real humans can actually log in.

Common mistake: Selecting elements by CSS class or position (button.btn-primary, nth-child(3)). These break the moment someone resigns a component. Use data-testid attributes instead — they're stable, semantic, and cheap to add.

---

🎭 Test Data Setup & Teardown

Bad test data is the #1 cause of flaky tests. Here are the patterns that actually work:

// Pattern 1: beforeEach / afterEach for full isolation per test
beforeEach(async () => {
  // Start fresh every test — nothing leaks from a prior run
  await db.raw('TRUNCATE orders, users RESTART IDENTITY CASCADE');
  await seedTestUsers();
});

afterEach(async () => {
  // Optional: extra cleanup for files, cache entries, emails sent, etc.
  await clearTestEmailInbox();
});

// Pattern 2: Transactions (rollback instead of truncate — faster)
beforeEach(async () => {
  await db.raw('BEGIN');
});

afterEach(async () => {
  await db.raw('ROLLBACK'); // nothing persists between tests
});

Mental model: Every test should start in a known, clean state and leave no trace. Tests that depend on the order they run in are time bombs. Treat each test like a tiny universe that begins and ends in isolation.

Why it matters: Flaky tests almost always come down to shared state. Test A creates a user. Test B expects no users. Depending on order, B passes or fails. Teardown makes order irrelevant.

Common mistake: Truncating tables in afterAll instead of beforeEach. If a test crashes mid-run, afterAll never fires and the next run starts with dirty data.

---

🐛 Flakiness — Causes and Fixes

Flaky tests fail sometimes and pass sometimes with no code changes. They're toxic. Teams stop trusting the suite and start ignoring failures — which defeats the entire purpose.

Mental model: A flaky test is not a test. It's a coin flip dressed up in test syntax. Fix flakiness immediately when it appears — the longer it lives, the more it poisons trust in the entire suite.

---

🛠️ Your Mission

You have a real Express API route: POST /api/tasks — it creates a task in the database and returns the created task. You also have a login-then-create flow in the browser.

Step 1 — Write the integration test:

// tests/integration/tasks.test.js
import request from 'supertest';
import { app } from '../../src/app.js';
import { db } from '../../src/db.js';

beforeEach(async () => {
  await db.raw('TRUNCATE tasks, users RESTART IDENTITY CASCADE');
  await db('users').insert({ id: 1, email: 'builder@hyve.org', name: 'Builder' });
});

afterAll(async () => {
  await db.destroy();
});

describe('POST /api/tasks', () => {
  it('creates a task and returns it with the assigned user', async () => {
    const response = await request(app)
      .post('/api/tasks')
      .send({ title: 'Ship it', userId: 1 })
      .expect(201);

    expect(response.body.title).toBe('Ship it');
    expect(response.body.userId).toBe(1);

    // Confirm it actually hit the DB — not just a mocked response
    const dbTask = await db('tasks').where({ id: response.body.id }).first();
    expect(dbTask).toBeTruthy();
    expect(dbTask.title).toBe('Ship it');
  });

  it('returns 400 when title is missing', async () => {
    await request(app)
      .post('/api/tasks')
      .send({ userId: 1 })
      .expect(400);
  });
});

Step 2 — Write the E2E test for the critical flow:

// tests/e2e/create-task.spec.js  (Playwright)
import { test, expect } from '@playwright/test';

test('authenticated user can create a task and see it in the list', async ({ page }) => {
  // Log in first
  await page.goto('http://localhost:3000/login');
  await page.fill('[data-testid="email-input"]', 'builder@hyve.org');
  await page.fill('[data-testid="password-input"]', 'testpass123');
  await page.click('[data-testid="login-button"]');
  await expect(page).toHaveURL('http://localhost:3000/dashboard');

  // Create a new task
  await page.click('[data-testid="new-task-button"]');
  await page.fill('[data-testid="task-title-input"]', 'My first real task');
  await page.click('[data-testid="save-task-button"]');

  // Confirm it appears in the task list
  await expect(page.locator('[data-testid="task-list"]'))
    .toContainText('My first real task');
});

Both tests together give you confidence that the full vertical slice — UI → API → database — works correctly end to end.

---

✅ You're done when…

• Your integration test suite passes every item on the Pre-Ship Checklist
• At least one integration test makes a real HTTP request to your app and queries the actual test DB to confirm persistence
• At least one E2E test covers the single most important user flow (login + primary action)
• beforeEach (or equivalent) seeds clean test data so tests are order-independent
• Your suite passes three times in a row with no flaky failures
• You can explain to a teammate what each layer of the pyramid catches and why you kept the E2E count small

---

➡️ Next: Test Doubles: Mocks, Stubs & Fakes. Build It Right, Or Don't Build It At All. 🏛️