How often does a “compatible” release break its users?

The short version

Our replay: stable releases published on Maven Central from 1 January 2024 to 8 November 2026 by 44 of the most-used Java libraries. A release is counted only when two independent tools, the Bumpwise engine and japicmp, both report a breaking change. Method and data.

This is the most recent measurement we know of for Java. The last large published study of Maven Central stops in 2018, and the best-known study of this kind, for Rust, dates from 2023. Both are below, with what they found.

What we did

• The sample was not chosen by us. We took the list of the most-used libraries on Maven Central that a research team published in 2025 with the Roseau paper: the top 100 by usage, filtered by its authors to Java libraries. Of its 66 artifacts, 46 contain code and had a stable release since 2024, and 44 of those had minor or patch releases.
• Every stable release was compared with the previous version. We compared the bytecode of the two published jars. Pre-releases were left out. A change in the first version number is a major release, in the second a minor release, anything else a patch release.
• Two tools had to agree. Each pair was checked by the Bumpwise engine (version 1.0.2) and by japicmp (0.27.0), an established open-source checker. Only releases flagged by both are counted.
• Internal packages were excluded. Classes in packages named internal, impl or shaded, and third-party code repackaged inside a jar, do not count.
• Breaking means binary or source incompatible. Either code compiled against the old version fails to link against the new one, or code written for the old version no longer compiles. Counting only binary incompatible changes, the figure is 9.8% instead of 13.0%.

The trend since 2019

We ran the same replay back to 2019 to see the direction: 3,096 minor and patch releases in total, 515 of them with a breaking change.

The rate fell from around one in five before 2024 to around one in eight since. It did not go away.

What broke

Nothing on this list is exotic. They are removals and changed signatures: changes that compile and pass the library's own tests.

Not every flagged change is a broken promise

A tool sees what is public in the bytecode, not what a project promises to support. Some projects keep classes public that they do not treat as stable API, and some do not follow semantic versioning strictly. Three libraries account for 48 of the 156 flagged releases: logback-core, spring-boot-autoconfigure and h2. Without them the rate is 10.0% instead of 13.0%. The median library is at 8.4%.

How far the two tools agree

Of the 1,201 releases, both tools flagged 156, Bumpwise alone flagged 3 and japicmp alone flagged 33. We examined the disagreements in the class files. The three that only Bumpwise flagged are false alarms, which we have logged as bugs. Most of what only japicmp flagged turned out to be compatible on inspection: for example an annotation element added with a default value, or a method that Guava kept available through a bridge class. Counting only agreement keeps both kinds of error out of the result.

What published studies found

Our replay is one measurement. Researchers have measured the same thing with other methods and other samples.

Java and Maven Central

Ochoa and colleagues compared the bytecode of 119,879 consecutive library upgrades on Maven Central, up to 2018. Of the 98,591 minor and patch upgrades, 19,868 contained at least one breaking change: 20.1%, or 37.9% of minor and 14.6% of patch upgrades. The rate had fallen from 67.7% in 2005 to 16.0% in 2018. A review of 97 studies published in 2026 still uses these figures as the reference for Java. Our 13.0% for 2024 to 2026, on a different sample, continues the same direction.

Two more recent studies measured the client side. Jayasuriya and colleagues updated the dependencies of 18,415 Maven projects: 11.58% of the updates contained a breaking change that affected the client, and almost half of those were non-major updates. In a second study they ran the clients' tests: 2.30% of updates changed behavior in a way that broke a test, most of them in non-major updates.

Rust

The authors of cargo-semver-checks ran it over every minor and patch release published since 2017 by the 1,000 most-downloaded crates: 14,389 releases. They found a violation in 464 of them (about 1 in 31) and in 172 of the 1,000 crates (more than 1 in 6). Their check was stricter than ours: each of the 3,062 violations was proven with a program that compiles against the old release and fails against the new one, and about 10,000 doubtful reports were thrown away. When they wrote to several maintainers, all confirmed the findings and most had not known. We found no newer run of this study.

Go and npm

For Go, Li and colleagues analysed 124,000 libraries and found that 28.6% of non-major upgrades introduce a breaking change, while the share of upgrades that follow semver rose from 63.7% in 2018 to 92.2% in 2023. For npm, Venturini and colleagues found that around 12% of a random sample of 384 packages were broken by a dependency during a minor or patch update.

A breaking change does not break everyone

Most breaking changes touch code that few people call. In the Maven Central study, 7.9% of the client projects that were checked would actually break. The Rust authors make the same point, and add: “with many projects having hundreds of dependencies, the odds are still concerningly high”. Our replay counts releases that contain a breaking change. It does not measure how many users each one reached.

Why nobody notices

These changes compile. The tests pass, because the tests were changed together with the code. The first people to find out are the users, after the release.

// 2.3.0
public int count() { ... }

// 2.4.0, released as a minor version
public long count() { ... }

On the JVM, a method is found by its exact descriptor, return type included. Every client compiled against 2.3.0 now fails with NoSuchMethodError, in code that nobody touched. The author sees a green build.

A careful reviewer does not reliably catch this either. The rules are long, different in every language and full of cases that look harmless: a wider parameter type, a new method in an interface, a renamed Kotlin parameter. The Rust authors reached the same conclusion about their own ecosystem:

“The maintainers are not to blame, and improved tooling is our best way forward.”Tomasz Nowak and Predrag Gruevski, 2023

The tooling that exists

Every ecosystem has a checker, and they are good. Almost all of them run in the build or in CI, after the change is written.

Bumpwise is our answer for the moment before the build. It is an IntelliJ IDEA plugin that compares the code you are editing with the last published release of your library. It highlights each change that breaks it, offers the fix that keeps it compatible or bumps the version when the break is intended, and calculates the next version. Many teams will want both: one check in the editor and one in CI.

Disclosure: Gilware makes Bumpwise, a paid plugin with a free edition for Java libraries on Maven Central. The replay on this page was run by us with the Bumpwise engine and japicmp. The other numbers come from published studies that we did not run.

What these numbers do not say

• Public is not the same as supported. Our replay counts changes to public classes outside internal packages. Annotations such as @Beta or @API(status = EXPERIMENTAL) are not taken into account, so some flagged changes are allowed by the project's own rules.
• The version number is trusted. Like the published studies, we assume a library follows semantic versioning when its versions look like it. Not every library promises that.
• The findings were not confirmed one by one. Two tools had to agree and we inspected the disagreements, but we did not build a failing client for each release as the Rust study did, and we did not ask the maintainers.
• Behavior is not checked. A method that keeps its signature and changes what it does is invisible to both tools. In that respect the numbers are too low.
• 44 artifacts are not Maven Central. They are the most used, and several belong to the same project: Spring, Jackson and JUnit account for many of them.
• The ecosystems cannot be ranked. Each study uses its own tool, its own definition of a breaking change and its own sample.

Sources and data

The result of every comparison in our replay can be downloaded: pairs.csv (one row per release, with what each tool reported) and summary.json (the totals on this page). Versions: Bumpwise engine 1.0.2, japicmp 0.27.0, run on 10 November 2026. Sample list: the replication kit of the Roseau paper [6].

1. Lina Ochoa, Thomas Degueule, Jean-Rémy Falleri and Jurgen Vinju. Breaking bad? Semantic versioning and impact of breaking changes in Maven Central. Empirical Software Engineering 27, 61 (2022). doi:10.1007/s10664-021-10052-y, arXiv:2110.07889.
2. Dhanushka Jayasuriya, Samuel Ou, Saakshi Hegde, Valerio Terragni, Jens Dietrich and Kelly Blincoe. An extended study of syntactic breaking changes in the wild. Empirical Software Engineering 30, 42 (2025). doi:10.1007/s10664-024-10563-4.
3. Dhanushka Jayasuriya, Valerio Terragni, Jens Dietrich and Kelly Blincoe. Understanding the impact of APIs behavioral breaking changes on client applications. Proceedings of the ACM on Software Engineering 1, FSE (2024). doi:10.1145/3643782.
4. Tomasz Nowak and Predrag Gruevski. Semver violations are common, better tooling is the answer. 7 September 2023. Discussion on Hacker News.
5. Juntao Chen, Tingting Bi, Yanlin Wang and Patanamon Thongtanunam. Breaking changes in software ecosystems: a systematic literature review. 2026. arXiv:2605.24397.
6. Corentin Latappy, Thomas Degueule, Jean-Rémy Falleri, Romain Robbes and Lina Ochoa. Roseau: fast, accurate, source-based API breaking change analysis in Java. ICSME 2025. arXiv:2507.17369, replication kit: zenodo.org/records/15536418.
7. Wenke Li, Feng Wu, Cai Fu and Fan Zhou. A large-scale empirical study on semantic versioning in Golang ecosystem. ASE 2023. doi:10.1109/ASE56229.2023.00140, arXiv:2309.02894.
8. Daniel Venturini, Filipe Roseiro Cogo, Ivanilton Polato, Marco A. Gerosa and Igor Scaliante Wiese. I depended on you and you broke me: an empirical study of manifesting breaking changes in client packages. ACM Transactions on Software Engineering and Methodology (2023). doi:10.1145/3576037.

Found a mistake, or a library we judged unfairly? Write to hello@gilware.com and we will correct it.