Skip to main content

Fuzz Testing and Property-Based Testing

Fuzz testing discovers bugs by automatically generating diverse, unexpected inputs and observing program behavior. While traditional tests check specific scenarios, fuzzing explores thousands of cases to find edge cases, crashes, and security vulnerabilities.

Overview​

Fuzz testing (fuzzing) and property-based testing are complementary approaches that generate test inputs automatically rather than manually specifying them. This automation enables discovery of bugs that developers wouldn't think to test - edge cases, boundary conditions, malformed inputs, and unusual combinations that break assumptions.

Traditional testing specifies exact inputs and expected outputs. You write "given input X, expect output Y." This is effective for known scenarios but misses unexpected cases. You can't write tests for bugs you don't anticipate.

Fuzz testing generates random or semi-random inputs, often intentionally malformed or extreme, then observes for crashes, hangs, memory corruption, or exceptions. Fuzzing excels at finding:

  • Security vulnerabilities: Buffer overflows, injection attacks, format string bugs
  • Robustness issues: Crashes from unexpected inputs, unhandled exceptions
  • Edge cases: Integer overflow, boundary conditions, null/empty inputs
  • Resource exhaustion: Memory leaks, infinite loops, excessive allocations

Property-based testing generates diverse inputs but focuses on verifying high-level properties (invariants) rather than specific outputs. Instead of "for input X, output is Y," you specify "for all inputs, property P holds." For example: "reversing a list twice returns the original list" or "serializing then deserializing data returns equivalent data."

Platform Applicability

Applies to: Spring Boot · Android · iOS

Fuzz testing primarily applies to backend services and native code handling complex input parsing, serialization, and security-critical operations.

Key differences:

AspectTraditional TestingProperty-Based TestingFuzz Testing
Input sourceDeveloper-specifiedFramework-generated (guided)Random/mutated
CoverageKnown scenariosWide range of valid inputsExtreme, invalid, malformed inputs
AssertionsExact expected outputProperties/invariants holdNo crashes, hangs, or exceptions
Best forBusiness logic verificationAlgorithmic correctnessSecurity, robustness, parsers
Failure analysisClear (known input/output)Automatic shrinking to minimal caseRequires debugging with fuzzer output

Both approaches complement traditional testing. Use traditional tests for core business logic, property-based testing for algorithms and data transformations, and fuzz testing for input parsing, serialization, and security-critical code. See our Testing Strategy for guidance on when to use each approach.

When to Use Fuzzing vs Property-Based Testing

Use property-based testing when you can define clear invariants and need reproducible tests in your CI pipeline. It integrates seamlessly with existing test suites and provides clear failure messages with minimized inputs.

Use fuzz testing when hunting for security vulnerabilities, testing parsers for malformed inputs, or validating robustness against unexpected data. Fuzzing excels at finding crashes and hangs but requires more setup and produces less readable test cases.

Use both for critical code like payment processing, authentication, or data serialization where both correctness (properties) and robustness (fuzzing) matter.

Core Principles​

  • Automate Input Generation: Let tools generate diverse inputs instead of manually creating test cases
  • Focus on Properties, Not Examples: Define invariants that should always hold rather than specific input/output pairs
  • Start Simple, Expand Coverage: Begin with basic properties and primitive types, then add complexity
  • Shrink to Minimal Failures: When tests fail, reduce inputs to the smallest reproducible case for debugging
  • Integrate Continuously: Run property-based tests in CI; run fuzzing campaigns regularly
  • Combine with Traditional Tests: Use both approaches together for comprehensive coverage

Property-Based Testing​

Property-based testing verifies that certain properties (invariants) hold across a wide range of automatically generated inputs. Instead of testing specific cases, you define universal truths about your code.

Core Concepts​

Properties are statements that should be true for all valid inputs. Common property categories include:

Invariant properties: Outcomes that never change regardless of input

  • "Sorting a list produces the same elements"
  • "Encrypting then decrypting returns original data"
  • "Parsing valid JSON never throws exceptions"

Round-trip properties: Operations that reverse each other

  • deserialize(serialize(x)) === x
  • decode(encode(x)) === x
  • fromString(toString(x)) === x

Idempotence properties: Running the operation multiple times has the same effect as once

  • sort(sort(x)) === sort(x)
  • normalize(normalize(x)) === normalize(x)
  • deduplicate(deduplicate(x)) === deduplicate(x)

Commutativity properties: Order of operations doesn't matter

  • add(a, b) === add(b, a)
  • merge(x, y) === merge(y, x) (for some merge operations)

Oracle properties: Compare against a simpler, trusted implementation

  • "Optimized algorithm produces same result as naive implementation"
  • "New caching layer returns same data as direct database query"

Shrinking is the process of reducing a failing input to its minimal form. When a property fails for input [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], the framework automatically tries smaller inputs like [1, 2, 3, 4, 5], then [1, 2], until it finds the smallest input that still fails, making debugging easier.

jqwik for Java​

jqwik is a property-based testing framework for Java that integrates with JUnit 5. It generates diverse inputs, executes tests hundreds of times with different values, and automatically shrinks failures to minimal reproducible cases.

Setup​

The jqwik library integrates with JUnit 5, allowing property-based tests to run alongside traditional unit tests. Add the dependency and configure the test task to recognize jqwik's test engine.

// build.gradle
dependencies {
    testImplementation 'net.jqwik:jqwik:1.8.2'
}

test {
    useJUnitPlatform {
        includeEngines 'jqwik'  // Enable jqwik test engine
    }
}

Basic Example​

import net.jqwik.api.*;

class StringPropertiesTest {

    @Property
    void reversingTwiceGivesOriginal(@ForAll String input) {
        String reversed = new StringBuilder(input).reverse().toString();
        String reversedTwice = new StringBuilder(reversed).reverse().toString();

        // Property: reverse(reverse(x)) == x
        Assertions.assertEquals(input, reversedTwice);
    }

    @Property
    void lengthIsPreservedWhenReversing(@ForAll String input) {
        String reversed = new StringBuilder(input).reverse().toString();

        // Property: length doesn't change
        Assertions.assertEquals(input.length(), reversed.length());
    }
}

By default, jqwik runs each property 1000 times with different generated values. The @ForAll annotation tells jqwik to generate values of that type. When a property fails, jqwik automatically shrinks the input to the simplest failing case - if it fails with a 50-character string, jqwik tries progressively shorter strings until it finds the minimal reproduction.

Real-World Example: Payment Validation​

class PaymentValidatorTest {

    // Property: validating twice gives same result (idempotent)
    @Property
    void validationIsIdempotent(
            @ForAll @BigRange(min = "0.01", max = "999999.99") BigDecimal amount) {

        PaymentValidator validator = new PaymentValidator();

        ValidationResult first = validator.validate(amount);
        ValidationResult second = validator.validate(amount);

        Assertions.assertEquals(first.isValid(), second.isValid());
        Assertions.assertEquals(first.errors(), second.errors());
    }

    // Property: positive amounts are valid
    @Property
    void positiveAmountsAreValid(
            @ForAll @BigRange(min = "0.01", max = "999999.99") BigDecimal amount) {

        PaymentValidator validator = new PaymentValidator();

        ValidationResult result = validator.validate(amount);

        Assertions.assertTrue(result.isValid(),
            "Positive amount " + amount + " should be valid");
    }

    // Property: negative or zero amounts are invalid
    @Property
    void nonPositiveAmountsAreInvalid(
            @ForAll @BigRange(max = "0.00") BigDecimal amount) {

        PaymentValidator validator = new PaymentValidator();

        ValidationResult result = validator.validate(amount);

        Assertions.assertFalse(result.isValid(),
            "Non-positive amount " + amount + " should be invalid");
    }

    // Property: serialization round-trip preserves data
    @Property
    void serializationRoundTripPreservesData(
            @ForAll @BigRange(min = "0.01", max = "999999.99") BigDecimal amount,
            @ForAll @StringLength(min = 3, max = 10) String currency) {

        Payment payment = new Payment(amount, currency);
        ObjectMapper mapper = new ObjectMapper();

        try {
            String json = mapper.writeValueAsString(payment);
            Payment deserialized = mapper.readValue(json, Payment.class);

            // Round-trip property
            Assertions.assertEquals(payment.getAmount(), deserialized.getAmount());
            Assertions.assertEquals(payment.getCurrency(), deserialized.getCurrency());
        } catch (JsonProcessingException e) {
            Assertions.fail("Serialization should not throw for valid payment: " + e.getMessage());
        }
    }
}

These properties run 1000 times each with different generated values, testing far more scenarios than manually written tests could cover. The @BigRange and @StringLength annotations constrain generated values to realistic ranges, ensuring tests focus on valid business scenarios rather than extreme edge cases that aren't relevant.

Custom Generators​

When built-in generators don't match your domain, create custom generators using Arbitraries:

class PaymentGenerators {

    @Provide
    Arbitrary<Payment> payments() {
        Arbitrary<BigDecimal> amounts = Arbitraries
            .bigDecimals()
            .between(new BigDecimal("0.01"), new BigDecimal("100000"))
            .ofScale(2);  // Two decimal places for currency

        Arbitrary<String> currencies = Arbitraries.of("USD", "EUR", "GBP", "JPY");

        return Combinators.combine(amounts, currencies)
            .as(Payment::new);
    }

    @Property
    void paymentAmountIsPositive(@ForAll("payments") Payment payment) {
        Assertions.assertTrue(
            payment.getAmount().compareTo(BigDecimal.ZERO) > 0,
            "Payment amount must be positive"
        );
    }

    @Provide
    Arbitrary<AccountTransfer> transfers() {
        Arbitrary<String> accountIds = Arbitraries
            .strings()
            .numeric()
            .ofLength(10);

        Arbitrary<BigDecimal> amounts = Arbitraries
            .bigDecimals()
            .between(new BigDecimal("1.00"), new BigDecimal("50000"))
            .ofScale(2);

        return Combinators.combine(accountIds, accountIds, amounts)
            .as(AccountTransfer::new)
            .filter(transfer ->
                !transfer.getFromAccount().equals(transfer.getToAccount())
            );  // Ensure source and destination differ
    }
}

The @Provide annotation creates reusable generators. Combinators.combine merges multiple generators, and .as() constructs your domain objects. The .filter() method ensures generated values meet constraints - here preventing transfers from an account to itself.

Custom generators enable testing realistic business objects rather than just primitives, making property-based tests more valuable for domain logic validation.

Advanced: Stateful Testing​

Property-based testing can validate stateful systems by generating sequences of operations and checking invariants after each step:

class BankAccountStatefulTest {

    @Property
    void balanceNeverGoesNegative(@ForAll("accountOperations") ActionSequence<BankAccount> actions) {
        // Start with an account with initial balance
        BankAccount account = new BankAccount(new BigDecimal("1000.00"));

        // Apply sequence of operations
        actions.run(account);

        // Invariant: balance is never negative
        Assertions.assertTrue(
            account.getBalance().compareTo(BigDecimal.ZERO) >= 0,
            "Balance must never be negative"
        );
    }

    @Provide
    Arbitrary<ActionSequence<BankAccount>> accountOperations() {
        return Arbitraries.sequences(
            Arbitraries.oneOf(
                depositAction(),
                withdrawAction(),
                transferAction()
            )
        );
    }

    private Arbitrary<Action<BankAccount>> depositAction() {
        return Arbitraries.bigDecimals()
            .between(new BigDecimal("1.00"), new BigDecimal("1000.00"))
            .map(amount -> account -> account.deposit(amount));
    }

    private Arbitrary<Action<BankAccount>> withdrawAction() {
        return Arbitraries.bigDecimals()
            .between(new BigDecimal("1.00"), new BigDecimal("500.00"))
            .map(amount -> account -> account.withdraw(amount));
    }
}

Stateful testing generates sequences like "deposit 100, withdraw 50, deposit 200, withdraw 300" and verifies invariants hold after each operation. This discovers bugs in state transitions that single-operation tests miss - for example, a race condition that only appears after specific operation sequences.

fast-check for TypeScript​

fast-check brings property-based testing to TypeScript and JavaScript. It generates diverse test data, runs properties hundreds of times, and shrinks failures to minimal reproducible cases.

Setup​

Install fast-check as a development dependency. Create a dedicated npm script to run property-based tests separately from traditional unit tests, making it easy to run them independently or skip them during rapid development iterations.

npm install --save-dev fast-check
// package.json
{
  "scripts": {
    "test": "jest",
    "test:property": "jest --testMatch='**/*.property.test.ts'"  // Run only property tests
  }
}

Basic Example​

import fc from 'fast-check';

describe('String properties', () => {
  it('reversing twice returns original', () => {
    fc.assert(
      fc.property(fc.string(), (input) => {
        const reversed = input.split('').reverse().join('');
        const reversedTwice = reversed.split('').reverse().join('');

        expect(reversedTwice).toBe(input);
      })
    );
  });

  it('length is preserved when reversing', () => {
    fc.assert(
      fc.property(fc.string(), (input) => {
        const reversed = input.split('').reverse().join('');

        expect(reversed.length).toBe(input.length);
      })
    );
  });
});

fc.assert runs the property 100 times by default (configurable). fc.property defines the test, and fc.string() generates diverse string inputs including empty strings, Unicode characters, and very long strings.

Real-World Example: API Response Handling​

describe('Payment API serialization', () => {

  // Property: round-trip serialization preserves data
  it('serialization round-trip preserves payment data', () => {
    fc.assert(
      fc.property(
        fc.record({
          id: fc.uuid(),
          amount: fc.double({ min: 0.01, max: 999999.99, noNaN: true }),
          currency: fc.constantFrom('USD', 'EUR', 'GBP'),
          timestamp: fc.date(),
        }),
        (payment) => {
          const serialized = JSON.stringify(payment);
          const deserialized = JSON.parse(serialized);

          expect(deserialized.id).toBe(payment.id);
          expect(deserialized.amount).toBeCloseTo(payment.amount, 2);
          expect(deserialized.currency).toBe(payment.currency);
        }
      ),
      { numRuns: 1000 }  // Run 1000 times for high confidence
    );
  });

  // Property: validation is consistent
  it('validates payment amounts consistently', () => {
    fc.assert(
      fc.property(
        fc.double({ min: -1000, max: 1000000, noNaN: true }),
        (amount) => {
          const validator = new PaymentValidator();

          // Run validation twice
          const result1 = validator.validateAmount(amount);
          const result2 = validator.validateAmount(amount);

          // Idempotence: same input gives same result
          expect(result1.isValid).toBe(result2.isValid);
          expect(result1.errors).toEqual(result2.errors);
        }
      )
    );
  });

  // Property: positive amounts pass validation
  it('accepts all positive amounts', () => {
    fc.assert(
      fc.property(
        fc.double({ min: 0.01, max: 999999.99, noNaN: true }),
        (amount) => {
          const validator = new PaymentValidator();
          const result = validator.validateAmount(amount);

          expect(result.isValid).toBe(true);
          expect(result.errors).toHaveLength(0);
        }
      )
    );
  });

  // Property: non-positive amounts fail validation
  it('rejects non-positive amounts', () => {
    fc.assert(
      fc.property(
        fc.double({ max: 0, noNaN: true }),
        (amount) => {
          const validator = new PaymentValidator();
          const result = validator.validateAmount(amount);

          expect(result.isValid).toBe(false);
          expect(result.errors.length).toBeGreaterThan(0);
        }
      )
    );
  });
});

The fc.record generator creates objects with specified properties, perfect for testing domain objects. The { numRuns: 1000 } option increases test iterations for critical code. Combining multiple generators (like fc.uuid(), fc.double(), fc.constantFrom()) creates realistic test data.

Custom Generators​

Build domain-specific generators for complex business objects:

// Custom generator for payment transfers
const transferGenerator = fc.record({
  fromAccount: fc.stringOf(fc.char(), { minLength: 10, maxLength: 10 }),
  toAccount: fc.stringOf(fc.char(), { minLength: 10, maxLength: 10 }),
  amount: fc.double({ min: 0.01, max: 50000, noNaN: true }),
  currency: fc.constantFrom('USD', 'EUR', 'GBP'),
  reference: fc.lorem({ maxCount: 20 }),
}).filter(transfer => transfer.fromAccount !== transfer.toAccount);

describe('Account transfers', () => {
  it('transfer amount is preserved', () => {
    fc.assert(
      fc.property(transferGenerator, (transfer) => {
        const processor = new TransferProcessor();
        const result = processor.process(transfer);

        if (result.success) {
          expect(result.debitedAmount).toEqual(transfer.amount);
          expect(result.creditedAmount).toEqual(transfer.amount);
        }
      })
    );
  });
});

// Generator for sequences of operations (stateful testing)
const accountOperationGenerator = fc.commands([
  fc.record({
    type: fc.constant('deposit' as const),
    amount: fc.double({ min: 1, max: 1000, noNaN: true }),
  }),
  fc.record({
    type: fc.constant('withdraw' as const),
    amount: fc.double({ min: 1, max: 500, noNaN: true }),
  }),
]);

describe('Account state', () => {
  it('balance is always non-negative', () => {
    fc.assert(
      fc.property(
        accountOperationGenerator,
        fc.double({ min: 1000, max: 10000, noNaN: true }),  // Initial balance
        (operations, initialBalance) => {
          const account = new BankAccount(initialBalance);

          operations.forEach(op => {
            if (op.type === 'deposit') {
              account.deposit(op.amount);
            } else if (op.type === 'withdraw' && account.balance >= op.amount) {
              account.withdraw(op.amount);
            }
          });

          expect(account.balance).toBeGreaterThanOrEqual(0);
        }
      )
    );
  });
});

The .filter() method ensures generated data meets business rules - here preventing transfers from an account to itself. The fc.commands() generator creates sequences of operations for stateful testing, discovering bugs that only appear after specific operation sequences.

Fuzz Testing​

Fuzz testing generates massive volumes of random, mutated, or malformed inputs to discover crashes, hangs, memory corruption, and security vulnerabilities. Unlike property-based testing which verifies invariants, fuzz testing primarily detects when programs fail catastrophically.

When to Use Fuzz Testing​

Fuzz testing is particularly effective for:

Parsers and deserializers: JSON, XML, Protocol Buffers, binary formats - any code that processes external data. Malformed inputs can cause crashes, infinite loops, or exploitable buffer overflows.

Input validation: Authentication, payment validation, data sanitization. Fuzzing discovers bypasses where unexpected inputs circumvent security checks.

Serialization/deserialization: Testing round-trip conversion (object → bytes → object) with corrupted data, truncated payloads, or invalid encodings.

File format handlers: Image processing, document parsing, archive extraction. Historically rich sources of security vulnerabilities.

Network protocol implementations: HTTP parsers, WebSocket handlers, API endpoints. Malformed requests can crash servers or enable denial-of-service.

Cryptographic code: Encryption/decryption, signing, hashing. Critical to test edge cases and ensure no crashes leak sensitive data.

Coverage-Guided Fuzzing​

Modern fuzzers use coverage-guided fuzzing: they monitor which code paths each input exercises, then mutate inputs that discover new paths. This feedback loop systematically explores the program, finding deep bugs that random testing would miss.

How it works:

  1. Instrumentation: Compiler adds probes to track which basic blocks execute
  2. Seed corpus: Start with valid example inputs (sample JSON, test files, etc.)
  3. Execution: Run program with each input, recording code coverage
  4. Mutation: Modify inputs that find new code paths (bit flips, byte insertions, splicing)
  5. Iteration: Repeat millions of times, building corpus of interesting inputs

Why coverage-guided fuzzing works: Random fuzzing struggles with code like if (input == "MAGIC_VALUE_12345") because random chance of generating the exact string is infinitesimal. Coverage-guided fuzzing notices that inputs starting with "M" reach slightly different code, then mutates those to eventually discover "MAGIC_VALUE_12345". This guided search finds bugs in complex parsers that pure random testing would never reach.

Jazzer for Java​

Jazzer is a coverage-guided fuzz testing engine for the JVM. It uses libFuzzer-style instrumentation to guide input generation toward discovering new code paths.

Setup​

Jazzer requires both a Gradle plugin (for build integration and instrumentation) and a JUnit dependency (for writing fuzz tests). The plugin instruments your code to track coverage, while the JUnit library provides the @FuzzTest annotation and fuzzing infrastructure.

// build.gradle
plugins {
    id 'com.code-intelligence.jazzer' version '0.22.0'  // Coverage instrumentation
}

dependencies {
    testImplementation 'com.code-intelligence:jazzer-junit:0.22.0'  // Fuzz test framework
}

Basic Fuzz Test​

import com.code_intelligence.jazzer.junit.FuzzTest;
import java.nio.charset.StandardCharsets;

class PaymentParserFuzzTest {

    @FuzzTest
    void fuzzJsonParsing(byte[] data) {
        // Convert random bytes to string
        String json = new String(data, StandardCharsets.UTF_8);

        try {
            // Parser should handle any input without crashing
            PaymentParser parser = new PaymentParser();
            parser.parse(json);

            // If parsing succeeds, verify basic invariants
            // (no assertion needed for fuzz tests - just don't crash)
        } catch (ParseException e) {
            // Expected for invalid JSON - not a bug
        } catch (IllegalArgumentException e) {
            // Expected for invalid payment data - not a bug
        }
        // Any other exception (NullPointer, OutOfMemory, etc.) is a BUG
    }

    @FuzzTest
    void fuzzPaymentValidation(byte[] data) {
        // Fuzz the payment validator
        String input = new String(data, StandardCharsets.UTF_8);

        PaymentValidator validator = new PaymentValidator();

        try {
            // Should never crash regardless of input
            ValidationResult result = validator.validate(input);

            // If validation passes, result should be usable
            if (result.isValid()) {
                assert result.getErrors().isEmpty();
            } else {
                assert !result.getErrors().isEmpty();
            }
        } catch (IllegalArgumentException e) {
            // Expected for completely invalid input
        }
    }
}

Jazzer generates random byte arrays and feeds them to your fuzz targets. The fuzzer monitors code coverage and mutates inputs that discover new execution paths. Unlike traditional tests, fuzz tests don't assert specific outputs - they verify the code doesn't crash, hang, or throw unexpected exceptions.

Real-World Example: API Request Fuzzing​

import com.code_intelligence.jazzer.junit.FuzzTest;
import com.code_intelligence.jazzer.api.FuzzedDataProvider;

class PaymentAPIFuzzTest {

    @FuzzTest
    void fuzzPaymentRequest(FuzzedDataProvider data) {
        // Use FuzzedDataProvider for structured fuzzing
        PaymentRequest request = new PaymentRequest();

        // Fuzz individual fields with appropriate types
        request.setAmount(data.consumeBigDecimal());
        request.setCurrency(data.consumeString(3));
        request.setAccountFrom(data.consumeString(20));
        request.setAccountTo(data.consumeString(20));
        request.setDescription(data.consumeRemainingAsString());

        PaymentProcessor processor = new PaymentProcessor();

        try {
            // Process should validate and reject invalid inputs gracefully
            ProcessingResult result = processor.process(request);

            // Verify invariants if processing succeeds
            if (result.isSuccess()) {
                assert result.getTransactionId() != null;
                assert result.getTimestamp() != null;
            }
        } catch (ValidationException e) {
            // Expected for invalid inputs
        } catch (BusinessRuleException e) {
            // Expected for rule violations
        }
        // Crashes, NPEs, or unexpected exceptions are BUGS
    }

    @FuzzTest
    void fuzzJSONDeserialization(byte[] data) {
        ObjectMapper mapper = new ObjectMapper();

        try {
            // Attempt to deserialize fuzzed data
            Payment payment = mapper.readValue(data, Payment.class);

            // If deserialization succeeds, object should be valid
            assert payment != null;

            // Round-trip should be consistent
            byte[] serialized = mapper.writeValueAsBytes(payment);
            Payment roundTrip = mapper.readValue(serialized, Payment.class);

            assert payment.equals(roundTrip);
        } catch (JsonProcessingException e) {
            // Expected for malformed JSON
        }
    }

    @FuzzTest(maxDuration = "10m")  // Longer fuzzing campaign
    void fuzzPaymentCalculations(FuzzedDataProvider data) {
        // Fuzz numerical calculations
        BigDecimal amount = new BigDecimal(data.consumeDouble(-1000000, 1000000));
        BigDecimal feePercent = new BigDecimal(data.consumeDouble(0, 100));

        FeeCalculator calculator = new FeeCalculator();

        try {
            BigDecimal fee = calculator.calculateFee(amount, feePercent);

            // Verify fee is reasonable
            assert fee.compareTo(BigDecimal.ZERO) >= 0;
            assert fee.compareTo(amount.abs()) <= 0;  // Fee can't exceed amount
        } catch (ArithmeticException e) {
            // Expected for invalid calculations
        }
    }
}

FuzzedDataProvider enables structured fuzzing by consuming the random byte array as typed values (strings, integers, decimals). This creates more realistic test data than raw byte arrays. The maxDuration parameter controls how long Jazzer runs - longer campaigns find deeper bugs but take more time.

Integration with CI/CD​

# .gitlab-ci.yml
jazzer-fuzz:
  stage: test
  image: eclipse-temurin:21-jdk
  script:
    # Run fuzz tests for 5 minutes each
    - ./gradlew fuzzTest --max-duration=5m
  artifacts:
    when: on_failure
    paths:
      - build/fuzz-results/  # Contains crash-inducing inputs
    expire_in: 30 days
  rules:
    - if: '$CI_PIPELINE_SOURCE == "schedule"'  # Run on schedule, not every commit

Fuzz testing in CI typically runs on a schedule (nightly or weekly) rather than on every commit, as comprehensive fuzzing takes hours. Failed fuzzing runs produce artifacts containing the exact inputs that caused crashes, enabling developers to reproduce and fix bugs.

fast-check for TypeScript Fuzzing​

While fast-check is primarily a property-based testing tool, it can also perform fuzz-style testing by generating extreme, malformed inputs:

import fc from 'fast-check';

describe('Payment parser fuzzing', () => {
  it('handles arbitrary strings without crashing', () => {
    fc.assert(
      fc.property(
        fc.string(),  // Generates any string including empty, Unicode, very long
        (input) => {
          const parser = new PaymentParser();

          try {
            const result = parser.parse(input);

            // If parsing succeeds, verify basic structure
            if (result) {
              expect(result).toHaveProperty('amount');
              expect(result).toHaveProperty('currency');
            }
          } catch (error) {
            // Parsing can fail, but should throw expected errors
            expect(error).toBeInstanceOf(ParseError);
          }
        }
      ),
      { numRuns: 10000 }  // Run many times for fuzz-like coverage
    );
  });

  it('handles malformed JSON gracefully', () => {
    fc.assert(
      fc.property(
        fc.string().filter(s => {
          try {
            JSON.parse(s);
            return false;  // Valid JSON - filter out
          } catch {
            return true;  // Invalid JSON - include in test
          }
        }),
        (malformedJSON) => {
          const parser = new PaymentParser();

          // Should not crash on malformed JSON
          expect(() => {
            parser.parse(malformedJSON);
          }).not.toThrow(TypeError);  // Type errors indicate bugs
          expect(() => {
            parser.parse(malformedJSON);
          }).not.toThrow(RangeError);  // Range errors indicate bugs
        }
      ),
      { numRuns: 5000 }
    );
  });

  it('handles arbitrary byte sequences', () => {
    fc.assert(
      fc.property(
        fc.uint8Array({ minLength: 0, maxLength: 10000 }),
        (bytes) => {
          // Convert bytes to string (may produce invalid UTF-8)
          const decoder = new TextDecoder('utf-8', { fatal: false });
          const input = decoder.decode(bytes);

          const parser = new PaymentParser();

          // Should not crash on any byte sequence
          try {
            parser.parse(input);
          } catch (error) {
            // Only expected errors are acceptable
            expect(error).toBeInstanceOf(ParseError);
          }
        }
      ),
      { numRuns: 10000 }
    );
  });
});

This approach uses fast-check's generators to create fuzz-like inputs (arbitrary strings, malformed JSON, byte arrays) and verifies the code handles them gracefully. While not as sophisticated as coverage-guided fuzzers like Jazzer or AFL, it's simpler to integrate into existing Jest test suites and provides good coverage for TypeScript applications.

Best Practices​

Start with Seed Corpus​

Provide example inputs to guide fuzzing toward interesting code paths:

// Jazzer seed corpus
@FuzzTest
void fuzzWithSeeds(byte[] data) {
    // Jazzer looks for seed files in:
    // src/test/resources/PaymentParserFuzzTestInputs/
    // Place sample JSON files there to guide fuzzing
}
// fast-check: use pre-defined examples
it('tests with realistic examples', () => {
  const examples = [
    '{"amount": 100, "currency": "USD"}',
    '{"amount": -50, "currency": "EUR"}',
    '{"amount": 0, "currency": "GBP"}',
  ];

  fc.assert(
    fc.property(fc.string(), (input) => {
      // Test random inputs
    }),
    { examples }  // Start with these before generating random inputs
  );
});

Seed corpora accelerate fuzzing by starting with valid or semi-valid inputs that reach deeper code paths. For a JSON parser, seeds should include various JSON structures (nested objects, arrays, edge cases). Fuzzers mutate seeds to explore nearby inputs, finding bugs faster than pure random generation.

Focus on Critical Code Paths​

Don't fuzz everything - prioritize:

  • Input parsing: JSON, XML, binary formats, protocol parsing
  • Authentication/authorization: Login, token validation, permission checks
  • Payment processing: Amount calculations, fee computation, currency conversion
  • Data serialization: Encoding/decoding, format conversion
  • Cryptographic operations: Encryption, signing, hashing

Set Realistic Timeouts​

Fuzz tests can run indefinitely. Configure reasonable limits:

@FuzzTest(maxDuration = "5m")  // Stop after 5 minutes
void fuzzPaymentProcessing(FuzzedDataProvider data) {
    // ...
}
fc.assert(
  fc.property(/* ... */),
  {
    numRuns: 10000,  // Limit iterations
    timeout: 5000,   // Timeout per test (ms)
  }
);

In CI pipelines, run fuzz tests on a schedule (nightly/weekly) with longer durations. During development, use shorter campaigns (1-5 minutes) for quick feedback.

Reproduce and Debug Failures​

When fuzzing finds a crash, it saves the crash-inducing input:

# Jazzer crash files
build/fuzz-results/crash-1234567890abcdef

# Reproduce the crash
./gradlew fuzzTest --reproduce=build/fuzz-results/crash-1234567890abcdef
// fast-check reports failing input in test output
// Copy the seed to reproduce:
fc.assert(
  fc.property(/* ... */),
  { seed: 1234567890 }  // Use seed from failure report
);

Always save crash-inducing inputs in version control as regression tests. After fixing the bug, add a traditional unit test with the exact input that caused the crash to prevent regression.

Combine with Other Testing Approaches​

Fuzzing complements but doesn't replace other testing:

Traditional unit tests: Verify specific business logic and edge cases Property-based tests: Check invariants with generated inputs Fuzz tests: Find crashes and security vulnerabilities Mutation testing: Validate test effectiveness Integration tests: Test component interactions

Use all approaches together for comprehensive validation. See our Testing Strategy for guidance on balancing different testing types.

CI/CD Integration​

Property-Based Testing in CI​

Property-based tests run in CI like traditional tests but with configured iteration counts:

# .gitlab-ci.yml
property-tests-java:
  stage: test
  image: eclipse-temurin:21-jdk
  script:
    - ./gradlew test -Pjqwik.tries=1000  # Run 1000 iterations
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH == "main"'

property-tests-typescript:
  stage: test
  image: node:22
  script:
    - npm ci
    - npm test -- --testPathPattern=".property.test"
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH == "main"'

Property-based tests should run on every commit and MR, just like unit tests. They're deterministic (given the same seed) and fast enough for continuous integration.

Fuzz Testing in CI​

Fuzz testing runs continuously or on schedule to discover new bugs over time:

# .gitlab-ci.yml
fuzz-scheduled:
  stage: fuzz
  image: eclipse-temurin:21-jdk
  script:
    - ./gradlew fuzzTest --max-duration=2h
  artifacts:
    when: on_failure
    paths:
      - build/fuzz-results/
    expire_in: 30 days
  rules:
    - if: '$CI_PIPELINE_SOURCE == "schedule"'  # Nightly or weekly
  allow_failure: true  # Don't block pipeline on fuzz failures

fuzz-quick:
  stage: test
  image: eclipse-temurin:21-jdk
  script:
    - ./gradlew fuzzTest --max-duration=2m  # Quick smoke test
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
  allow_failure: true

Quick fuzz tests (1-5 minutes) run on MRs to catch obvious crashes. Long fuzz campaigns (hours) run on schedules to find deeper bugs. Use allow_failure: true to avoid blocking pipelines while still surfacing issues.

Continuous Fuzzing with OSS-Fuzz​

For open-source projects, OSS-Fuzz provides continuous fuzzing infrastructure:

Benefits:

  • Runs 24/7 on Google infrastructure
  • Automatically files bugs when crashes are found
  • Provides detailed crash reports and reproducers
  • Integrates with GitHub/GitLab for issue tracking

Setup requires creating a project configuration in the OSS-Fuzz repository. Google's infrastructure then continuously fuzzes your project, finding bugs over weeks or months that short CI runs would miss.

Troubleshooting​

Fuzz Tests Timing Out​

Symptom: Fuzzer hits timeout without finding new coverage

Causes:

  • Infinite loops in code under test
  • Very slow code paths
  • Fuzzer stuck in unproductive mutation cycle

Solutions:

@FuzzTest(maxDuration = "10m", maxCycles = 100000)  // Adjust limits
void fuzzTest(byte[] data) {
    // Add timeout to code under test
    Callable<Void> task = () -> {
        parseInput(data);
        return null;
    };

    ExecutorService executor = Executors.newSingleThreadExecutor();
    Future<Void> future = executor.submit(task);

    try {
        future.get(1, TimeUnit.SECONDS);  // 1-second timeout per input
    } catch (TimeoutException e) {
        future.cancel(true);
        // Timeout is expected for some inputs
    }
}

Property-Based Tests Failing Intermittently​

Symptom: Test passes locally but fails in CI, or vice versa

Cause: Non-deterministic behavior (time-based logic, random values, external dependencies)

Solutions:

@Property
void deterministicTest(@ForAll String input) {
    // Mock time-dependent behavior
    Clock fixedClock = Clock.fixed(Instant.parse("2024-01-01T00:00:00Z"), ZoneId.of("UTC"));
    PaymentService service = new PaymentService(fixedClock);

    // Test is now deterministic
}
it('uses deterministic behavior', () => {
  fc.assert(
    fc.property(fc.string(), (input) => {
      // Mock randomness
      jest.spyOn(Math, 'random').mockReturnValue(0.5);

      // Test is now deterministic
    })
  );
});

Shrinking Takes Too Long​

Symptom: When test fails, framework spends minutes shrinking the input

Cause: Complex nested data structures or large inputs

Solutions:

fc.assert(
  fc.property(/* ... */),
  {
    endOnFailure: true,  // Stop shrinking on first failure
    // Or limit shrinking attempts:
    numRuns: 1000,
    maxSkipsPerRun: 100,
  }
);
// jqwik: configure shrinking
@Property(shrinking = ShrinkingMode.OFF)  // Disable shrinking
void fastFailingTest(@ForAll String input) {
    // ...
}

For debugging, disable shrinking to see the original failing input immediately. Re-enable shrinking once you understand the failure pattern.

Summary​

Key Takeaways:

  1. Property-Based Testing Generates Inputs: Define properties (invariants) that should hold for all inputs; the framework generates diverse test cases automatically
  2. Fuzz Testing Finds Crashes: Discover security vulnerabilities, robustness issues, and edge cases by testing with malformed and extreme inputs
  3. Use jqwik for Java: Integrates with JUnit 5, provides powerful generators, automatically shrinks failures to minimal cases
  4. Use fast-check for TypeScript: Brings property-based testing to JavaScript/TypeScript with Jest integration
  5. Use Jazzer for Java Fuzzing: Coverage-guided fuzzing for JVM applications; finds bugs in parsers, validators, and security-critical code
  6. Focus on Invariants: Properties like round-trip serialization, idempotence, and commutativity reveal bugs that example-based tests miss
  7. Combine with Traditional Testing: Property-based and fuzz testing complement unit tests, mutation testing, and integration tests
  8. Run in CI: Property-based tests run on every commit; fuzz tests run on schedules for deep exploration
  9. Prioritize Critical Code: Focus fuzzing on parsers, input validation, authentication, payment processing, and serialization
  10. Save Crash Inputs as Regression Tests: When fuzzing finds bugs, add the crash-inducing input as a traditional unit test

For comprehensive testing strategies combining property-based testing, fuzzing, mutation testing, and traditional approaches, see our Testing Strategy guide.

Further Reading​

Internal Documentation:

External Resources: