Defining an API management strategy
Reduce development overhead and time to deployment by standardising how teams design, launch and manage APIs in your organisation.
If you build or manage any number of APIs, it’s best practice to have an API management strategy in place. This guidance outlines some areas you should consider when making decisions about how to manage APIs in your team or organisation.
Principles for your API strategy
Build foundations early
If you have APIs, you should use an API gateway. Even a single API will benefit from the standardised tools an API gateway provides, by not having to develop and deploy those elements of functionality. If you build more APIs later, having an API gateway already in place will mean you will not have to build those tools again, reducing duplication of effort for future teams.
It might be difficult to apply this model if:
- you already have APIs in production which have their own security or monitoring
- your APIs have each been developed using different methods and technologies
Consider how you can put in place standardised patterns, processes and governance your teams will use to build and deploy APIs. You should focus on principles and frameworks rather than specific technologies, and consider how detailed and rigid these should be. For example, you might apply strict rules for security, while allowing more flexibility in other areas.
You can read more about using an API gateway in the section of this guidance called ‘Managing operations with an API gateway’.
Keep the future in mind
When putting together your API strategy, it’s important to consider what your organisation’s future needs are likely to be, not just think about the first few APIs.
Maintain a high level view of your APIs. You should have a way to judge the maturity of your APIs and determine how much resource you need to invest in them. Since APIs at different stages of their life cycle require different levels of investment, having a coherent strategy allows you to identify and meet these needs.
Design your strategy to meet user needs
When making decisions about API management, you must meet user needs as well as organisational goals. Make sure you understand how development teams in your organisation work so your policies and processes are not counter-productive. For example, overly strict criteria for assuring or publishing APIs might make teams reluctant to try new solutions that do not fit the pattern, which could discourage innovation.
User research will help you gain insight into how teams are producing and consuming APIs, or what technologies they are using or considering using.
A robust API management process will help build credibility and trust with users of your APIs. It’s important to demonstrate your APIs are well designed and well supported. For example, being clear about how you retire APIs will reassure external developers they will not suddenly lose access to an API without notice.
Define roles for API management
Depending on the size and structure of your organisation, you might have several teams involved in building and maintaining APIs. Your API management strategy can help set out things like:
-
who owns different parts of the API life cycle
-
what skills or roles API teams should include
-
how ownership changes as an API goes into service
For example, it’s common for a central team to run the API gateway, and therefore have control over service levels and capacity. You should then consider how that responsibility might interact with the design and delivery responsibilities of an API owner, like versioning.
Having a clear structure for escalating issues will save a lot of time and energy in the future. Support from senior management or stakeholders can help formalise organisational structures and policies.
What API management strategy can help with
An API management strategy can:
- provide visibility of all your APIs, which helps people discover them and promotes reusing existing technology
- allow you to standardise common design patterns
- help you automate administrative and operational tasks
- provide a central place for your users to access API documentation and support
- provide data and metrics to help you understand your API’s performance and usage
- help you implement and maintain good security and data protection practices
Your API management strategy should cover the full life cycle of APIs, from design to retirement.
Designing APIs
You should establish design guidance to help maintain consistency across your APIs. It’s difficult to manage APIs that are not consistent, so the more your APIs follow the same design principles and practices, the easier it will be to manage them at scale.
The level of guidance you provide will depend on the size and structure of your organisation. For example, guidance might be different for an environment where API design follows strict standards, and an organisation using many different technologies and approaches.
Review and challenge assumptions
Organisations often develop their API strategy based on the first few APIs they built, where choices teams made in response to specific problems are then applied across very different areas. If you have an existing API strategy in place, consider what assumptions you might be relying on and make a plan to review your approach regularly.
User research can help you test these assumptions and define needs. You might design your research to help you understand:
- what technologies are in use, or might be in the future
- the different levels of maturity of development teams in the organisation
- whether the APIs produced are primarily internal or external facing, or a mixture
- what service level agreements (SLAs) APIs should meet and whether different SLAs are appropriate for different use cases
Bring developers into the design process
API design guidance reduces the time and effort developers need to spend making decisions, by providing a framework for them to work from.
You should also explain design choices. Being open with the reasoning behind the guidance increases developer trust and helps get them on board with the process.
Use standards
Technology standards like OpenAPI can help increase confidence in software decisions. Standards can also give credibility to your strategy as they reflect widely accepted ways of doing things.
It’s helpful to specify use cases for different API standards. As an example, you might require using REST for microservices and GraphQL for more data intensive processes.
You should also consider what guidelines make sense for specific technical details. For example, using a consistent data format, or requiring that all APIs follow the same approach to versioning.
Make the design process clear and easy to use
Setting out decisions in a central place helps development teams get started on projects more quickly and with less effort. You should keep these guidelines in a document store or repository that is easy for people to find and access.
Various tools are available to store and share API specifications. Many tools have an open source version which allows you to try them out with little to no cost or commitment. These tools often form part of an API management suite that also includes a developer hub and an API gateway but can also be used as standalone design or governance modules.
Put your API through assurance and assessment
Assurance is a governance process to make sure APIs meet various organisational standards. For example, assurance will help teams make sure they are building APIs to follow the design guidelines of your organisation.
Your assurance assessment programme should be appropriate for the teams who need to go through it. Make it clear what the benefits of going through assurance are so that teams see it as a worthwhile exercise to help them improve their APIs. If your assurance process is too rigid, or requires too much effort for not enough benefit, teams might try to avoid it. This will weaken your organisation’s governance and can impact the overall quality of your organisation’s APIs.
Test and validate your APIs
Testing is the process of making sure the APIs function properly, and is usually an automatic process which validates the code. Testing should be an ongoing process, and regular validation of the API should be part of a monitoring scheme.
Usually, testing is done by an automated tool against the specification, sometimes known as the contract. Contract testing should include checking the:
- specification against internal policies to make sure that APIs submitted to the developer portal are compliant with internal standards
- API to make sure its responses match the specification and that deployed APIs are behaving correctly in production
Putting APIs into production
Deploying your API involves more than just pushing code to production. There are a number of things you should put in place to support the publishing process and to provide a good developer experience.
Document your APIs
Your API documentation is the starting point for development teams looking to use your API.
You should work with a technical writer from the start of your project. If you do not have technical writing resources in your organisation, you can contact the cross-government technical writing community for help.
When documenting APIs, you should include:
- conceptual information to introduce the API
- practical information such as a ‘getting started’ guide
- detailed reference documentation
You can use an API specification to generate and auto update reference documentation for your API as you build and iterate it.
Developer portal
A developer portal, also sometimes called a developer hub, is where external developers can access everything they need to set up and manage their integration with your API. This usually includes:
- documentation for your API
- developer registration and authorisation
- self service tools for things like renewing API keys and changing their password
- a test environment - for example a mock API, a ‘sandbox’ and test users
- issue reporting and ticket support
You can also use your developer portal to gather internal metrics about your API programme. For example, you might want to measure how quickly developers are able to set up a test version of an API. This is sometimes known as “time to live” or “time to 200” and is a useful metric to measure how easy your API is to integrate. It can also help you identify where there might be pain points for users.
Managing operations with an API gateway
The everyday operations of API service management are usually handled by an API gateway. Generally, an API gateway service acts as a proxy for APIs, and allows central services like access control and security to be provided for all the APIs using the service.
At a minimum, an API gateway should provide:
- user and application authentication
- rate limiting and throttling inbound requests
- logging and reporting
Many organisations do not put an API gateway into production until they have several APIs in use. However, you should consider implementing one at the very beginning of an API programme because this will allow you to:
- standardise how APIs use central services such as network access, security and authentication, throttling and rate limiting, logging and monitoring
- lower developer costs and reduce time to deployment, since you will not need to rebuild these services for each new API
- require minimum standards of security, access control and monitoring, while providing an easy way for developers to meet them
Adding an API gateway means you will need to deploy and manage another service, but it reduces complexity in managing your APIs and introduces a consistent front end for services.
There are many different API gateway products available, both open source and proprietary. They will often include elements for a developer portal or a catalogue.
Making your APIs easy to find
APIs should be visible and discoverable in a central and up-to-date place. You can use an API catalogue to help users discover your APIs, find out what they do and how to use them. An API catalogue also helps your organisation keep an inventory of what APIs it manages.
Keep API discovery separate from your developer portal
Most API management tooling includes features for implementing an API gateway, the developer portal and an API catalogue, making it a convenient way to use one tool for all of these things. However, you should not rely solely on a developer portal to make your APIs discoverable.
There are cases where an API might not be appropriate to sit on the gateway or the developer portal, but should still be discoverable in a catalogue. This might be because the API is still in experimental stages, does not meet standards, or because the team is not ready to hand over management of the API.
Keeping the catalogue separate from the other parts of your API management tool helps people discover all your APIs without restricting them to those that meet certain criteria.
This helps:
- promote an environment where innovation can happen in the open
- avoid teams developing in silos to get around rules
- improve transparency and reduce duplication of effort
Consider who will be looking for APIs in your organisation
Most organisations have 3 levels of access for their APIs:
- private - internal and kept entirely within the organisation
- partner - shared externally with a limited set of trusted users and partners
- public - open to external users either directly or through a self-service sign up platform
Consider how best to help each group of users find your APIs. You could do this with separate catalogues, but you should consider the cost and effort it would require to maintain these. You can also use access control to restrict visibility of APIs, or details of the APIs, to registered users at different levels of authentication.
You should aim to expose as much detail as possible to all users. Even for sensitive data sets, you should expose basic details of an API with information about how to get access. This helps developers understand the value of the API and start interacting with it while they wait for access approval.
Check with your internal security teams about what level of exposure is acceptable for each API. You might also find it helpful to review the metadata model for the government API catalogue. This provides a basic discovery framework that does not increase the vulnerability of published resources.
Link your catalogue to the government API catalogue
Your internal catalogue should update the government API catalogue, either programmatically or with manual submissions. This catalogue is a centralised directory of government APIs, and any external APIs (whether inter-departmental, restricted or public) should be listed on there.
Contact the API Catalogue team at api-catalogue@digital.cabinet-office.gov.uk for more information and help with adding your APIs or connecting your internal catalogue.
Taking your API out of service
You must plan how to take your APIs out of service when they are no longer needed. This is often referred to as retirement, deprecation, sunsetting or decommissioning.
Taking an API out of service suddenly or with poor support can significantly impact the services integrating that API. It can also damage your organisation’s credibility.
Being prepared for how you’ll retire APIs helps:
- build trust with users - these are the external developers using your APIs
- users plan for any changes they need to make to their services
- protect your organisation’s credibility
It might be useful to put together an API retirement workflow or checklist for your organisation to help teams follow a consistent process. Below is an example of what this might look like.
- Use analytics to support your case for deprecation.
- Publish a blog post to explain the reasons and offer alternatives where possible.
- Add a deprecation notice to documentation with the date it will happen.
- Disable sign ups in self-service to stop new users accessing the service.
- Email subscribed users with the date of deprecation - as the date approaches, emails should get more frequent.
- Use Sunset or Deprecated HTTP headers, with a link to the documentation and blog post.
- Wait and see how usage changes - make sure it’s dropping and contact any remaining active users.
- Agree an acceptable number of active users at retirement.
- Keep the API in retired status for a while - this could be months or years.
You should publish your retirement policy somewhere external developers can see it, for example on your developer hub or in API documentation. This will help reassure users that you provide a good level of support throughout the API life cycle.