From feature request to API design

Users often are the main source of feature requests when building open-source libraries. A request might be too specific to be implemented as is. This leads to a feature that only benefits a single user while your original target is wider.

This post describes the process of going from a feature request to an API design fitted for many users. It is illustrated by the library Medium Zoom that I created and maintain. This post is more suited for library authors rather than product developers.

The term “users” in this blog post refers to the users of library APIs (i.e. developers).

Keep the problem you’re solving in mind

A library should stay focused and shouldn’t solve too many problems at once. A way to not get too scattered is to have a philosophy in mind.

For example, Medium Zoom’s philosophy is to reproduce the same user experience as the image zoom on the Medium blog platform in vanilla JavaScript.

If you start accepting one more feature, you’re set up for another. And another. And one more. You should seek for solving the user problem using the existing API. If this is not possible and that the demand keeps increasing, think about broader use cases to create a more generic solution. To do so, you need to dig the why.

This allows setting boundaries on what your project should allow and what your project shouldn’t allow. It draws a line on what the library is offering. I like to think of a framework as a consumer of your code, while you’re the consumer of a library. When developing a library, you should aim at minimizing your API surface area.

Understand your users’ needs

It’s easy to misunderstand a user request if you jump too fast from your understanding of their problem to a solution.

It happens that users suggest their solution to the problem in a GitHub issue. This is often misleading because the solution is too specific to their own need.

By default, Medium Zoom offers a clean UI with only an image zoomed, and no other elements on the overlay. Back in 2017, a user wanted to add a button to close the zoom on the library. Their solution was to add a button option on the overlay itself. This feature would have only been useful for their use case.

After digging what the user exactly wanted to achieve, I came up with the concept of templates. Templates allow you to zoom in your container, and to display other elements in addition to the image.

Besides solving this user’s problem, it solved many others. For instance, it is now possible to reproduce the Facebook zoom or the Dropbox Paper zoom.

Don’t stop at understanding how users want to use your API, ask them why they need this feature.

Don’t be too specific

Building a library is different from building an application because you need to foresee how users will use the API. This is a very hard guess to make but it’s a skill that you can grow over time.

I considerer an API to be good when it gives developers enough power to build their feature on top of it without being too permissive (otherwise the abstraction is unnecessary in the first place).

Creating an API that is too specific is known as overfitting in statistics:

In statistics, overfitting is “the production of an analysis that corresponds too closely or exactly to a particular set of data, and may therefore fail to fit additional data or predict future observations reliably”. — Wikipedia

The user asking for a button option in Medium Zoom was trying to solve their problem. A button option would have been too specific and only few people would have benefited from it.

The more you add to your API, the more options your users need to learn and the more confusing it gets. Eventually, you increase your API surface area. A useful pattern to give flexibility to developers is Inversion of control.

Start with documentation

To not waste time getting caught up in unnecessary technical details, start documenting the usage of the new feature.

When designing Medium Zoom, I spent a lot of time writing documentation upfront. This has driven the API design and helped make sure that the API covers the cases I had planned. Besides knowing what tools to give to the API users, I knew where to draw the line to not be too permissive.

Having this process before starting the development makes you think about the API that you wish to use. It brings you to a better mental framework to start getting into code. This wishful API doesn’t take into account all the technical difficulties that bias the API when you start programming. It leads to a more objective API.

This reasoning makes you forget about what you already know and leads to creative ideas. It comes close to first-principles thinking:

First-principles thinking is one of the best ways to reverse-engineer complicated problems and unleash creative possibility. Sometimes called “reasoning from first principles,” the idea is to break down complicated problems into basic elements and then reassemble them from the ground up. It’s one of the best ways to learn to think for yourself, unlock your creative potential, and move from linear to non-linear results. — Farnam Street

Don’t rush to the implementation, start documenting the usage of your library. This technique is called README-Driven Development.

Tips

It’s often better to start small before creating a complete API. I wouldn’t recommend covering use cases that haven’t proved to be necessary. This often happens when you want to solve problems that haven’t matured yet and therefore the wrong problems.

Removing an existing feature from an API has more consequences than adding the feature in the first place. In other terms, it’s easier to add a feature than to remove one. Developers and library authors themselves become dependent on this feature.

Solutions to getting rid of a feature are either to create non-stable APIs to gather some feedback, or to bump your library to a new major version. Communicating why you’re dropping a functionality helps users understand the move. Yet, you’re doomed to explain to them how to achieve the same result with an alternative API.

Conclusion

To sum up, when thinking about creating or extending your API:

• Have a clear understanding of what problem you want to solve
• Understand why a feature is needed at its core
• Try to create a generic solution
• Don’t rush to the implementation; start with documentation

Those are the lessons I learned working on some open-source libraries, including Medium Zoom as mentioned in this article, as well as the InstantSearch suite at Algolia (InstantSearch.js, React InstantSearch, Vue InstantSearch, and Angular InstantSearch).