Skip to content

Observability for API Testing

Tamir Hyman
By Tamir Hyman
6 min read
API Observability
Observability for API Testing

If you’re like me, part of your job is to write API tests for your system. When you have a small number of APIs, the task is relatively easy. It becomes complex when you’re dealing with hundreds of APIs. 

From one developer to another, here are some of my thoughts on how to approach this.

The root problem, of course, stems from developers adding features quickly. Faced with time pressures, they add these features as patches, which can make your APIs act strangely. 

Prioritizing APIs for Testing

Priority image

The first task is deciding which APIs to test - prioritization. The only way to really test a system’s APIs is knowing its business logic and prioritizing and testing those APIs according to that business logic.

Any prioritizing mechanism can fit; it all depends on which application is being tested. When I get started, I start by thinking to myself “I want to test the most “X” APIs.”

Here are a few of the approaches I use, depending on the project. 

  1. Most used - Make sure that your most used APIs are tested with positive, negative, and regression tests to improve reliability.

  2. Most sensitive – A more security-guided approach. Test the APIs that handle sensitive data to make sure you don’t have security issues.

  3. Most buggy – Test your most problematic APIs. These APIs might be more error-prone and testing them will save you a lot of debugging in the future.

  4. All APIs – It may sound obvious, but it’s worth noting. This approach works well in a small organization, or on a small number of APIs, but isn’t scalable for the long term. In my opinion, it’s better to have good tests on prioritized APIs than “empty” tests on all APIs.

So, how should you prioritize?  

Start with Observability

The key here is observability. A system with user-facing APIs never behaves as expected. The developers might think that one API is more “X” than another, but user interactions almost always prove them wrong.

To prioritize effectively, we must see what happens in our system. This can be done through logging, monitoring systems, observability systems, or any other tool you have in mind that provides data on what is happening in our system. 

Once you’ve looked under the hood and determined which APIs you should prioritize, it’s time to start testing. 

observability - glass

The Challenges of Testing APIs

Testing an API is not always as easy as it seems. Unlike Unit Tests, an API isn’t always standalone. You can’t always directly call it, and your input and output are much more complicated. For example:

  1. A common REST practice is to work on resources. There is usually an API to create a resource, and multiple APIs are deployed to edit and use the resource. Assume for example you have an API that creates a shop `POST /api/shops`. This one is easy to test – you can just call it and create a shop.Now, though, you want to test the API that adds an item to the shop `POST /api/shops/{shop_id}/item`. This one is tricky since you need to create a shop before you add the item.This may sound easy, but in real systems with a lot of resources it gets complicated very fast. Some APIs require multiple resources that must be created before even attempting to access the API.

  2. Another resource-related problem is that your input/output for the API might depend on the input/output of previous APIs.

Typing fast

For example, in our shop API, each shop has a type, and only items of that type can be added to the shop. If we create a fruit shop and attempt to add a potato in the test, we will fail. However, to test this API, you will need to set the stage first by creating a shop and defining the types of items that can be added.

  1. APIs might behave differently depending on the input parameters, and you can’t always figure that out from looking at the code. Most APIs have multiple parameters, and their behavior changes according to:

    1. What parameters were passed (required/optional)

    2. The actual values of the parameters – sometimes different values cause a completely different logic. 

Different behavior can also cause the result to be different and change the assertions of the test.

Observability helps solve these challenges. All these challenges are further complicated by the fact that if you are tasked with testing APIs in a large system, you probably weren’t the person who wrote them. This makes it nearly impossible to completely understand the logic and rationale of the APIs’ behavior. 

With correct observability, though, you can get answers for almost any of the problems I mentioned. Monitoring user flows can help you prioritize which logic inside the API to test, and how to test it.

In our example, if you observed that most users created fruit shops and added about 10 different items, you can deduce the flow of the test as well as the desired parameters and assertions.

Another aspect of observability is the relationship between the different APIs. This can help achieve higher coverage of specific APIs/resources. For example, if we know all the different APIs that use shop_id, we can cover most of the functionality of a shop owner. This can also help us in cases where we don’t know which resources are required by a specific API.

With all this information in place, we can write our tests.

programmer for API testing

Are We Finished Yet?

With the tests written, it’s easy to get trapped into thinking the job is done. That is, sadly, not the case. 

The hardest part of writing tests is actually rewriting them. APIs continuously change and tests constantly break - it’s the circle of life for developers. Here’s why:

  1. New APIs are added to support new functionality – requiring us to perform the whole cycle for them too.

  2. APIs are changed to add/remove functionality – this may cause our tests to fail or be irrelevant.

  3. Priorities might change. New data is added to the system which might affect the initial prioritization of the APIs.

The job of an API tester never ends. It’s a constant struggle between writing tests and maintaining old tests. But, with the correct tools and methodology, it can be much simpler. Continuous analysis of relevant data can help focus efforts and improve efficiency.

Want to auto-generate API test skeletons and data to save valuable developer time? Contact us!