Introduction to API Interaction

A look at the Web API ecosystem and an overview of your place within it.

Scroll down...

Content

Resources

Comments

APIs are the lifeblood of the web, pumping precious JSON to all of its distal servers. If you want your application to be a good, upstanding web citizen, you'll probably want it to associate with a few APIs. Your apps can't do everything themselves, you need to focus on what sets your app apart while delegating to other tools and services (often web APIs) when expedient.

APIs are just interfaces that applications share so that your code can talk to theirs (in one important sense, any public method is an API), but what we're talking about here are third-party, HTTP-based services where you can request data from remote servers and get it packaged up the way you want for your own use.

We'll focus on the practical here because our goal is to get you on the right foot to perform more complex API integrations.

Registering

Almost all APIs will require you to register yourself and your application with the API provider. This allows them to track and monitor your usage.

There is typically a free tier of usage, and it will cost you if you're going over your limits. These limits are expressed in terms of total MB downloaded, number of requests, or number of transactions (for instance).

S3 storage and request pricing:

Amazon S3 Pricing Tiers

Usually, even if you're a user of the API provider's consumer-facing service, you will need to separately go to their developer portal and set up a developer account.

When you set up your account, you'll get an API key. This key uniquely identifies your application to the API provider, and you will usually send it along with every request. You'll typically also get a "secret key" or something that functions like one. Whereas the API key is usually public, the secret key is an added layer of security that's required for the more secure APIs.

When authenticating with secure APIs, the secret key is used to generate a cryptographic token that authenticates any requests you make. While it doesn't matter much for the first assignment, in the future you will need to make sure you don't include this secret key in your Git repo! There are various approaches using libraries and environment variables you will later learn to keep that precious API key out of other people's hands.

Types of API Requests

Most APIs require a different type of "security clearance" for different categories of requests. We'll cover a few of the more common high-level use cases here and what's involved with each.

Simple GET Requests

You can usually make innocuous requests (like asking Twitter for a few Tweets) with straightforward and unauthenticated GET requests. You can make these from any app, the command line or a browser tool like Chrome's Rest Client. These types of API requests are usually severely rate limited to incentivize you to sign up.

These requests are often behind the scenes when you make embedded API calls, for instance when you see a website displaying Tweets inside their posts. These hit the Twitter API, but aren't really considered traditional API calls and don't usually subject the hosting site to rate limitation.

API Key Requests

The next layer of complexity is making requests that include your API key. Some APIs require this for every request, while others allow you to use the previously-mentioned simple GET requests anonymously. Often, they'll allow very targeted GET requests (e.g. pulling a single photo or listing a user's Photo IDs) to pass anonymously but require that anything more complex (e.g. batches of photos or a full index of photos) be key-authenticated.

The whole point of using the API key is so the provider can link you to your usage, so expect these kinds of requests to be limited by the API's rate and sometimes pricing tiers. This also allows the provider to enforce their Terms of Use so you don't go building a competitive site based on their data.

CRUD-dy Requests

CRUD (Create/Read/Update/Destroy) operations that involve anything beyond simple READs will require a more advanced form of authentication. Usually, instead of just submitting single one-off requests like before, you'll need to authenticate each request individually. We'll cover this more during later lessons, but essentially you negotiate with the API provider and create a temporary access token based on your secret key. That token is used to validate the request before it goes through.

Obviously, these requests are subject to their rate and pricing tiers.

Requests on Behalf of Users

Applications aren't always just mindlessly pulling and pushing data. A use case that has greatly expanded over the past several years is making requests on behalf of a User. You've seen this if you've ever clicked "ok" on "Allow this application to make requests on our behalf" or "Allow this application to post to your Facebook timeline" or "This application requests access to your contact list" form.

Web request on behalf of

In all of these cases, your application negotiates with the API provider (e.g. Facebook or Google) so it can submit requests on behalf of the user. This starts when the user successfully authenticates with the API (e.g. signs in with their Facebook account). When that happens, your app is given an access token specifically for that User. You can use that token (along with your own secret key) to make requests on that user's behalf.

These requests will often fall under a different usage tier (e.g. a per-user tier) instead of being bucketed along with your normal application. If you can imagine, a large consumer app serving as a middleman for millions of Photos or Tweets or Posts wouldn't be viable if their API usage was all bucketed to the application provider. By spreading it out on a per-user basis, developers have a much greater flexibility to use these APIs in a widespread fashion.

"Sign In with Facebook" Authentication

Many applications have skipped implementing sign-ups of their own and instead rely on their users authenticating using third-party services like Facebook or Google or Github. You've no doubt seen these in the form of "Sign up with Facebook" buttons. The whole idea is that the user only needs to remember their Facebook password and doesn't have to entrust your application with it.

When you click here...

Oauth Login step 1

...it connects via "OAuth" to Facebook:

Oauth Login step 2

You can consider this a special case of the previous example. The basic flow is essentially the same — the user signs in with Facebook and then Facebook provides you with an access token for that user. Your authentication system will then essentially use this authentication token in place of a user password. When the user needs to sign in again, Facebook will ask for their password again and provide you with the token. If the user is already signed in with Facebook in their browser, this is often a one-click process.

Front End vs Back End APIs

APIs don't ultimately care whether you are querying them from deep within your Express app or by using JavaScript code in the browser. Certain types of requests should only be made from the server because they involve passing secure data around. Many APIs are accessed from the backend for this reason.

In this section, we're focusing on the back-end code you already know, but be aware that front-end JavaScript has its own set of approaches to accessing APIs and is a fast-maturing field. We will return to this when you get to the appropriate units.

API Restrictions

Every site, in addition to rate limiting their API, will have terms of use. This usually covers what you're allowed to do with the data you gather. For instance, they probably aren't going to allow you to store a whole bunch of their data in your own database (harvesting). They also probably won't allow you to do anything SPAMmy or which will hurt their user's experience (posting constantly to their Facebook). Common sense should be your initial guide, but consult the TOU documentation for specifics. Basically, if you'll be hurting their user's experience or taking their precious data, you can't do it.

It should be noted that any time you develop using someone else's API, you're putting yourself at their mercy. If you're using it to the point where you pay for data, they'll love you. If you get too successful and become competitive with them, they've got you by the neck and will cut you off or demand exorbitant fees. Just be cognizant of those long-term risks.

Documentation

Your first step after deciding to integrate with an API is to read through their documentation. It will (if it's decent) tell you everything you need to know about how to set up your account, how to authenticate, and what methods are available to you.

Reading through API docs is a crucial skill for a web developer, so we're going to leave you some leeway here to really figure this stuff out.

Versions

We've also discussed versions before but it's worth noting again — many APIs have gone through several versions so make sure you're referencing the correct documentation. Blog posts or Stack Overflow posts may also reference the older version, so check the date and be a bit suspicious of everything you read.

You can see the version (2.2), for instance, in the URL for this call to Stack Overflow's API:

response = HTTParty.get('https://api.stackexchange.com/2.2/questions?site=stackoverflow')

API Data

You often have a choice of which type of data to return from an API. The de-facto standard these days is JSON, so you'll commonly see JSON APIs. These are usually RESTful, which means that for a particular resource (like a Tweet) you'll have structured ways to ask for particular pieces of data or collections of data (like all Tweets from you in the last 3 days) back. They might also directly return JavaScript objects, XML objects, or (between web services) SOAP objects. In any case, you or the library you're using will need to deserialize these objects to make them useful.

Handling the Response Object

Let's say you've made a request to a third-party API. Now what?

You've officially interfaced with the Wild West, and any number of things could happen. This is where your nice, neat, perfectly executing web app has to start handling the reality of dealing with network requests.

The request might have timed out. Or your API key may be expired. Or you might get back a strange error ("Oh sorry, the callback you specified doesn't match the domain you're on..."), or the response could contain partial data, etc. Unfortunately, your user doesn't care about this potential variability. You'll need to build logic which can handle all possible outcomes from these APIs.

If connecting to and authenticating with the API is the first pain-in-the-butt (PITA) problem, handling all the possible outcomes of this connection is definitely the second. We'll cover some things here, but this is one of those areas where you'll often just need to go plunging through error messages to diagnose the issue.

"Sandbox" Connections

Many of the more established APIs will provide you with a "Sandbox" environment that you can make development requests of. It can be useful for avoiding real user data while you're developing. Sometimes these can be as much of a pain to connect to as the production area, so your mileage may vary.

Wrapping Up

Hopefully, this high-level API overview has you thinking about all the right components so we can begin diving into actual API use cases. We'll cover some simple ways to access web APIs in pure JavaScript. You'll definitely get a good taste of the range of ways that APIs can be both useful and challenging.



Sign up to track your progress for free

There are ( ) additional resources for this lesson. Check them out!

Sorry, comments aren't active just yet!

Next Lesson: Approaching a New API