Building it yourself: How to implement metered billing
Suppose that PixelMate offers the following off-the-shelf plans that users can sign up for:
- A monthly plan at $500/mo, with 1000 included images. After 1000 images, you’re charged $0.01/image for the first 100, and $0.008 thereafter.
- A more generous annual plan with an upfront yearly payment of $5000, with 1500 included images each month. After the included monthly images, we’ll charge $0.005 per image.
In order to charge customers today, we use Stripe Invoicing. PixelMate’s BizOps team goes through a monthly close process where each of the 80 customer’s charges are tallied and invoices are sent out by the 4th day of the month. Because PixelMate is growing, it has started to get painful, so we’re off to make this better with some automation.
Existing infrastructure: a simple starting point
Let’s assume a simple starting point for the lifecycle of a request to PixelMate.
- The system receives an API request on application servers from an end user, with an API token. The request contains an image which needs to be processed, and uses the API token to identify the customer.
- Images are uploaded to S3, processed by async workers, and the final result is stored in our data model.
- In order to notify the end user that processing is complete, the async worker logs an event to a Kinesis Firehose stream, which is then consumed by a system which fires off a completion notification via email.
Using this existing infrastructure, we’ll sketch out how to architect a full-featured metering and billing system relying on Stripe Billing.
Sketching out our events pipeline
Products and prices in Stripe Billing
Since we already use Stripe’s invoices, let’s set up Stripe Billing to handle the automation. Stripe Billing feels like a natural tool to reach to, since Stripe is already being used in our organization and their documentation advertises what looks like comprehensive support for the usage-based billing use case. When a user signs up for PixelMate, we’ll receive a signup request and the plan that they’re intending to use — this should then create a subscription in Stripe Billing so we start charging the customer based on their plan.
Accordingly, we’ll create the following in Stripe:
For monthly subscribers, we'll subscribe to both the base fee and the overage, setting up the invoices with both line items — we’re all set in terms of billing behavior for that option.
Unfortunately, the annual case isn’t straightforward: Stripe Billing won’t let us set up an invoice with both an annual and monthly fee — different cadences aren’t supported. To get around this, we’ll have to create two Stripe subscriptions for each customer, one to each product. This isn’t ideal, since our customers will see two separate invoices, and failures could mean that these subscription lifecycles aren’t always aligned. Nevertheless, we’re set up to start charging both the platform fee and overage.
Connecting usage to Stripe Billing
As we’re evaluating this design, there’s a troubling edge case: what happens if we miss a reporting window? This could happen because our containers are down for an extended period of time, Cloudwatch has an outage, events aren’t being delivered to Redshift, or there’s simply a bug in our reporter. Stripe does not allow reporting a usage record for a subscription item once the billing period has already passed — this would lead to missed usage.
Even barring cases where our usage reporter is down for unforeseen reasons, this can happen in the normal course of business. For example, when our usage reporter is active around the end of the month, it may very well fail to report usage for a subset of customers in the previous period, because the month boundary has gone by. It’s possible we may be able to report it for the following month instead, but there will be plenty of downstream issues here (we’ll lose the ability to audit since our invoices don’t match Redshift timestamps, our revenue recognition will be wrong, and so on).
When discussing this edge case with our product manager and finance analyst, we decide to prioritize the guiding principles of invoice accuracy and invoice clarity. We should ensure that users pay exactly for what they consume, and there is full transparency and clarity on what’s being billed. We collectively decide that the edge case around billing period boundaries is not one we can accept from a business perspective, and go back to the drawing board for a working solution to report usage to Stripe Billing.
Stripe Billing setup (take 2)
- Our PixelMate data model is now the source of truth for the customer’s tiering price configuration, since Stripe just manages the base monthly fee.
- The webhook consumer that we build has to now implement the tiering logic to translate a quantity from Redshift into one or more line items that need to be added to Stripe. Even with a simple multi-tier graduated price, this is error-prone and sensitive business logic!
Regardless, this solution meets our correctness criteria even if it falls a bit short on simplicity or separation of concerns.
Providing mid-period visibility
With some work, we’ve now set up a system that allows us to bill our customers accurately and on-time for their incurred usage. How might we now provide both internal teams and our end users some visibility into their upcoming invoice? This would be a valuable part of the PixelMate admin console and allow our product and growth team to better understand how user adoption is tracking.
This isn’t possible using Stripe’s upcoming invoice API with the approach we’ve settled on so far, since Stripe doesn’t know about usage until the invoice is about to finalize.
We’ll resort to just hooking up a tool like Metabase to our Redshift instance — at least this will get us a view of usage in a month and some simple day-by-day visualizations for our internal teams. With some more engineering work, we’ll be able to combine this with our application datastore to build an internal dashboard of the customer’s upcoming invoice.
Changing prices for new customers
Six months down the road, we’ve gotten feedback from customers that our tier boundaries aren’t set exactly correctly. Pricing is not a “set it and forget it” exercise, so our product partner is keen on us iterating on it. Our new goal is to make a slight modification to each tier, increasing the price of the Annual tier slightly and tweaking tier allocations.
We will roll this change out to only existing customers, as we want to monitor impact to our conversion and upgrade funnel before rolling it out to new customers.
When we want to change pricing on existing customers, we won’t be able to rely on metadata from the subscription creation. Instead, we’ll have to add more business logic to our codebase that identifies which invoices for a subscription should use the old and new tier amounts.
The bottom line
Building metered billing with Stripe is doable, but it’s tricky to get right and even more complex to maintain. As an engineering or product organization, billing infrastructure is understandably not an area of sustained engineering investment. If you’re facing this challenge and want to get time back for your core engineering work, we'd love to help. Orb handles streaming and deduplication of events, flexible in-product queries, and supports your stakeholders through the full revenue journey. You’re invited to explore our sandbox to get a sense of the product.
(By the way, if plumbing work like this excites you, we’re hiring!).