Within the last decade, there’s been an impressive transition from monolith applications into microservices. While pushing the industry forward and enabling more robust progress, that transition also had a side effect of increasing the number and reliance on APIs.
In fact, nowadays, many companies have adopted an “API-first” methodology, which emphasizes the natural growth in APIs and considers them more than just a method of accessing services. They are now viewed as the modern means of communication between an organization and their customers and within an organization itself.
Our fairy tail does not end here though as, like many improvements, a new set of problems arise. To overcome them, there’s been an emergence of practices, methodologies, and software solutions to fill the gap. One practice that is currently gaining traction is APIOps - a practice that incorporates governance, automation, and observability.
In this blog, we go over the concept of API governance, which is the practice of creating API policies and continuously validating that API producers adhere to them. Unfortunately, governance is quite hard to put into practice as its scope goes far beyond just “checking API designs”.
The challenges dev teams face around API governance
No convention enforcement = lack of reusability
When you have a limited number of APIs, it’s easy to enforce conventions and reusability of components between your endpoints. But as you increase the quantity of APIs needed, you’re likely to have different developers working on them, even across different teams, or maybe across different sites around the globe. Keeping a convention to your APIs becomes a hard task now, even though it’s just as important.
You may even have style guides and Confluence articles that explain how your developers should build, iterate, and push APIs. But without an enforcer, all you can do is hope your developers will adhere to the style requirements. You can choose to have a dedicated person double-check each API as well, but that requires a review period, delays releases, and takes autonomy out of development.
Having consistent conventions allows you to better find anomalies and errors. It makes it easy to find the API that doesn’t follow the “rules” as the differences will catch any developer’s eye. It also gives both the API producers and consumers a better understanding of the API and allows them to navigate through API lists easily.
The documentation barrier
The importance of proper and up-to-date documentation is enormous when it comes to APIs. Think about all of your API consumers; they include external (outside of your business) users and internal (your own developers) users.
Consider a new hire, for example, who needs to work with an undocumented API. What should they do first? Most likely, they will have to either spend time “exploring” each API (via reading code, trial and error, etc.) or they’ll need to ask an experienced developer to guide them through it. Even an high level developer from a different team or a different site will face the same issue for undocumented APIs they didn’t develop.
Now, think about your customers or external collaborators who need to face the same issues, but unfortunately, they cannot ask an experienced developer or explore your code.
We see this same issue arise when forgetting to update the documentation upon changes. If you made a change to any one of your APIs, forgetting to document that change will result in many API consumers facing the same issues as mentioned here.
Breaking changes - an integrators worst nightmare
Each API is a contract between the consumer and the producer. When you make a breaking change- for example, adding a new mandatory field in the API or removing a field from the response - you are likely to cause errors for your consumers.
If users are not aware of those changes, they are likely to only find out about them the hard way - AKA when their API calls start to fail.
Simply updating the documentation is unlikely to solve this issue. Developers need visibility into how other teams are changing and updating the APIs they are responsible for in order to move the product in the right direction.
Putting things practically, after opening a new PR, for example, developers should be able to see an inventory of the APIs from their system in the current state.
As changes are made and as development progresses, developers should have the insights and be immediately notified whenever a breaking change occurs by comparing the current swagger file to the base commit of the PR branch.
It shouldn’t stop at just identifying the breaking change, though. Developers need observability into what consumers are impacted by these changes by comparing it to their latest production or dev deployment snapshot.
You can’t secure what you don’t manage
As developers, we tend to apply strict security policies only for “external” APIs (APIs that are exposed to customers), and we consider internal APIs as less relevant for security because they are less likely to be targeted.
Unfortunately, that statement doesn’t hold true anymore. According to the IBM Security X-Force Cloud Threat Landscape Report of 2021, two-thirds of incidents started with misconfigured APIs. APIs are the gateways to data, so when access control is not properly implemented across all APIs, attackers can gain unauthorized access.
The security of your customers’ data should be top priority and should be treated carefully. Besides legal consequences, they trust your business to ensure their data will never be exposed or leaked.
It is crucial to understand which internal services access which APIs, and make sure all connections are necessary and logical. For example, consider a payments microservice and a notification service, which has nothing to do with payments. How would you react if the notification service was accessing an API of the payments microservice? You would have tried to understand why that is happening, and then probably eliminate that connection?
Scaling applications, a bittersweet risk
A scaling application is everyone’s dream. More users, more customers, more revenue.
But scaling comes at a cost and should be planned for in advance. Building and executing for scale is a part of the API design process. It’s not enough to expose secure APIs; you must also verify that public APIs can scale and satisfy client requests at all times and during peak demand.
Of course, it may not need to be at the level of Netflix or Facebook, but it should be a foundation from which you can grow on.
The importance of monitoring
Last but not least, there's the challenge of monitoring APIs. It is a fundamental tool that enables developers to understand scalability issues, find security vulnerabilities, alert on a sudden bad customer experience (for example, after a version release, if an API has a 90% increment in failed results, does it mean we have a breaking change?), and in general, give us the ability to know if our product and business functions well.
Your API governance checklist
A strong API governance strategy prioritizes 3 key areas: control, agility, and observability.
Control: This is all about establishing rules and regulations regarding how APIs should be designed and operated - for example, naming conventions, pagination type, etc. You also need to clarify how teams should work, including guidance on documenting, testing, securing, and monitoring APIs. It’s also important to monitor your teams’ compliance to the established rules.
Agility: There are various business areas and functions inside an organization, and each may have slightly different policies and rules. For example, a team in charge of developing payment APIs may have tougher controls than those who do not deal with sensitive data. Good API governance should be flexible enough to take into account the uniqueness of each business area and enable the relevant teams to choose the types of controls that apply to them, guaranteeing agility and adequacy in different business contexts.
Observability: API observability keeps you informed on the current state of your APIs. With API observability, you can see how well your APIs comply with your governance policies, and detect outliers and violations of the policy easily. Without API observability, implementing new policies, and verifying that existing ones are implemented correctly, is borderline impossible.
Best practices for creating an API governance strategy
As we’ve just established, API governance is important to protect your business from a variety of API-related risks. To create an efficient API governance policy, consider these tips:
1. Clarify your API goals and identify stakeholders
Clarifying your API goals will help you grasp how your APIs will evolve and what kind of rules and policies you should apply to them. Once you understand your goals, it’s also important to identify the stakeholders. Stakeholders can be customers, external collaborators, developers, product owners, and so on. Once you know who they are, you need to understand their workflows to craft a better API governance strategy.
2. Explore and create an inventory of your APIs
Governance is impossible when you don’t know what you’re governing. To implement API governance, you need an overview of what APIs are available in your organization and their current state. This includes both APIs in development and in production.
To accomplish such a big goal, automate the process of creating an API inventory. The inventory should contain the APIs’ description and formats, security policies in use, and documentation, as well as changelogs for different versions, schemas, and more.
3. Create internal API specs
The majority of API inconsistencies are due to disparities in design, naming conventions, and schemas. To ensure engineers stay on the same page, make sure you have clear style guides and API policies about how APIs should be created and built. Style guides should include architectural style, API description format, naming conventions, acceptable and unacceptable terms, error handling, and how to version APIs.
This will drive consistency and incentivize teams to build on established standards.
4. Automate policies and rule checks
Even with an extensive style guide written up in confluence, as your team grows, it becomes more difficult to manually review and ensure adherence to standards. Manual review is time consuming and requires a lot of effort. It also increases the feedback loop, unlike automated checks that provide immediate feedback to your team.
In addition, not every company can dedicate a developer to reviewing APIs per release. That time could be used creating new features or testing quality. Your best bet is to automate the review process as much as possible with an OpenAPI “style checker”.
5. Ensure API governance policies are met before deployment
Urgent deadlines and fast delivery are some of the reasons teams avoid governance before deployment. However, if API governance requirements are not met in time, noncompliance will go undetected, potentially until the offending APIs cause bugs, slowness, or even expose security exploits. Then, it will be too late and expensive to fix. If your company adheres to proper API governance prior to deployment, you’ll benefit from quick feedback, and you can prevent these changes before they are met by your customers.
6. Apply governance at all stages
Controls and governance are frequently viewed as roadblocks in a typical setup when governance is added at a later stage of an API lifetime. It’s important not to wait until APIs are in a production environment. Rather, you should apply API governance from the design level, through the developing phase, code quality checks, deployment to production, and until an API is declared deprecated and removed from the code base and deployment.
7. Embrace API observability
It’s a fact; your APIs will grow over time. That’s why API observability must be embraced as it gives you visibility into changes made between versions and across environments.
It allows you to look deeper into your APIs to understand their status, so you can discover new issues and failure modes and define proper governance and management.
The benefits of proper API governance
Most enterprises have seen what a lack of governance can do and, as a result, they understand how important it is. API governance takes a significant amount of planning, investment, and iteration. However, a good governance strategy offers many benefits, such as:
API governance ensures optimal visibility, discoverable APIs, properly organized teams, and cross-team collaboration. With observability, you can examine changes across your APIs and understand how API design is done. It simplifies API behavior patterns and dependency analyses, as well as defines how they are used by and dependent by consumers.
Better change management
APIs can stop working or disappear, or a breaking change could be introduced. Robust governance ensures you have proper change management to handle these issues. You’ll be able to catch and remediate breaking changes before they impact customers.
Design work changes the look and feel of an API, as well as how it will be used. Because every visible work done on an API involves design, it's very easy to introduce inconsistencies in large teams if there is no standard way of doing things. With API governance, you can drive uniformity in style, contract, and schema, and enforce system-wide adherence.
API development involves making frequent technical decisions. When you make quality decisions quickly, product delivery and iteration becomes easier. Governance improves the decision-making process of engineering teams, irrespective of how large they are. With API governance, we will be able to track violations and possible problems closer to the development rather than while deployed to production and used by customers. Thus, reducing time and cost for fixing the violation.
One of the promises of the digital transformation is that products will be released, tested, and changed more quickly in order to provide more value to end users. If you know what changes are coming and what their impacts will be, agility is bound to improve.
API governance and observability allow engineers to live up to that promise. In particular, API governance can help engineers get the information they need to proactively react, without impacting existing customers.
Overall, API governance has proven to be a critical component for every engineering department in large enterprises that operate and scale APIs that expose business capabilities.
Seekret’s API-first platform empowers developers with revolutionary eBPF-powered APIOps through automation, governance, and observability.
Our goal is to provide development teams with the proper methodology and tooling needed to become an API-first company by seamlessly integrating with their existing infrastructure.
With Seekret, you can design, develop, deploy, and monitor your APIs while leveraging governance and automation to ensure your teams have all the resources they need to succeed.