Calibre tracks performance budgets. AuditJet tracks what a missed budget costs you.
Calibre is one of the most respected web performance monitoring platforms on the market. This page explains how it works, what it does well, where it falls short, and how AuditJet approaches the same problems differently.
Background
What is Calibre?
Calibre is a continuous web performance monitoring platform built specifically for engineering teams. Founded in 2016 by Karolina Szczur and Ben Schwarz, it emerged from the Australian web community at a time when performance tooling was fragmented — teams relied on on-demand tools like WebPageTest or manually running Lighthouse audits, neither of which gave a longitudinal view of how performance changed over time. Calibre's core insight was that performance needed to be treated like uptime: something you monitor continuously, not just check occasionally.
The platform targets frontend-focused engineering teams, particularly at product companies and digital agencies where web performance is a competitive differentiator. Its design philosophy is "engineering-first": Calibre assumes its primary users are developers and performance engineers who want precise, actionable data rather than executive summaries. The integration surface reflects this — native connections to GitHub, Slack, and Jira mean that performance data lives inside the tools engineers already use every day.
Calibre uses Lighthouse-based synthetic testing under the hood, running scheduled scans from multiple global test locations. Each test simulates a real browser loading the page under defined device and network conditions and captures the full suite of Lighthouse metrics: Largest Contentful Paint (LCP), Cumulative Layout Shift (CLS), Total Blocking Time (TBT), Speed Index, Time to Interactive, and others. Because the test conditions are consistent across runs, the resulting data is highly comparable — you can draw a meaningful line chart of LCP over the past six months and trust that changes in the line reflect real changes in the page rather than test variability.
Perhaps Calibre's most significant contribution to the performance industry has been its work to normalise "performance culture" — the idea that speed should be a shared engineering value, not a one-time project. The team has produced influential writing, conference talks, and tooling around making performance part of every team's workflow, not just something the performance specialist thinks about. This cultural positioning has made Calibre something of a reference point in the web performance community, and it shows in the product's depth and attention to developer experience.
The platform positions itself at the intersection of monitoring and developer tooling. It is not a business intelligence tool or an executive dashboard — it is, deliberately, a technical instrument. Understanding this positioning is essential to evaluating whether Calibre is the right tool for a given team.
Technical foundations
How Calibre works
To understand Calibre, it helps to understand the category of monitoring it belongs to: synthetic monitoring. Synthetic monitoring runs automated, scripted tests from controlled server-side environments on a defined schedule. In contrast to real-user monitoring — which collects performance data from actual visitors using browser APIs — synthetic tests are reproducible. Every run uses the same device profile, the same network conditions, the same starting state. This reproducibility is what makes synthetic tests valuable for regression detection: when the line on your LCP chart jumps up overnight, you can be confident the change reflects something that happened in the code or infrastructure, not a change in the mix of real users visiting the site.
The core concept that Calibre built its product around is the performance budget. A performance budget is a threshold you set on a specific metric: "LCP must not exceed 2.5 seconds," or "total page weight must stay under 500 KB," or "JavaScript bundle size must not grow by more than 10 KB per deploy." Budgets transform abstract performance scores into enforceable engineering constraints. When a deploy causes a metric to exceed its budget, the system raises an alert — and if that budget is wired into a CI/CD pipeline, it can block the deploy from proceeding entirely. This is the performance equivalent of a failing test suite: the signal to the team is not "something looks slow," it is "something specific broke a specific contract."
Calibre's test profiles are another important concept. A test profile is a combination of device type (mobile or desktop), network speed (3G, 4G, broadband, cable), and geographic test location. A single URL can have multiple profiles — for example, a product team might monitor their checkout page as a mobile user on a simulated 4G connection from London and as a desktop user on broadband from Sydney. Each profile produces its own independent time series of metrics, so regressions can be observed to affect mobile more than desktop, or European users more than Australian users. This granularity is particularly valuable when investigating CDN configuration or network-dependent resource loading.
Under the hood, Calibre runs Google Lighthouse, the same open-source auditing tool that powers Google PageSpeed Insights. The advantage of this is familiarity: every metric Calibre reports is the same metric engineers already understand from PageSpeed, and improvements in Lighthouse itself flow through to Calibre automatically. The difference is timing — PageSpeed Insights runs on demand, when someone manually checks a URL. Calibre runs on a schedule, producing a continuous record without human intervention.
For teams that want to catch regressions before they reach production, Calibre offers GitHub Actions integration. When a pull request is opened, Calibre can automatically test a preview deployment and compare the results to the baseline. If a performance budget is exceeded — for example, LCP increases by more than 200ms compared to the main branch — Calibre posts a failing status check on the PR. This blocks merge until the regression is addressed or a team member explicitly overrides the budget. The result is a genuinely useful enforcement mechanism: performance regressions are caught at code review time rather than discovered weeks later from a dashboard.
Platform capabilities
Key features of Calibre
Performance budgets with PR blocking
Calibre's performance budget system is one of the most configurable in the market. Budgets can be set at the level of individual Lighthouse metrics (LCP, CLS, TBT, Speed Index) or at the level of resource sizes (total bytes, JavaScript bytes, image bytes). You can set absolute thresholds ("LCP must not exceed 2.5s") or relative ones ("LCP must not increase by more than 10% versus the previous deploy"). The practical consequence of relative budgets is that teams do not need to revisit and update budget configurations as the site evolves — the budget tightens or relaxes proportionally to the baseline.
When wired into a GitHub PR workflow, budget failures appear as a named check on the PR. The check surfaces the specific metric that failed, the before and after values, and the budget threshold. A developer seeing this knows immediately that they shipped something that increased LCP by 340ms, that the budget is 200ms, and that the most likely culprits are the resources loaded on that branch. This specificity is what makes the integration genuinely actionable rather than noise.
Build-over-build comparison
Calibre maintains a timeline of every deploy, making it possible to trace a performance change to a specific commit. The interface allows engineers to annotate deploys with a version label or commit hash, so the time series chart shows not just a line moving up and down but a line with labelled inflection points. This correlation between code changes and metric changes is one of the most practically useful features in the product — it compresses the investigation time when a regression appears from "look through the past two weeks of commits" to "the regression started on deploy v2.14.1."
Filmstrip and waterfall analysis
For diagnosing why a page is slow, Calibre provides two complementary views. The filmstrip shows a frame-by-frame visual timeline of the page loading, making it easy to identify when the page becomes visually complete and what causes the layout to shift. The waterfall chart shows every resource the page requests, ordered by time, with timing breakdowns for DNS lookup, TCP connection, SSL handshake, and content download. Together, these tools allow engineers to answer specific questions: "Which resource is blocking first render?" and "Is the hero image arriving before or after LCP is recorded?" This level of diagnostic depth is typically only available in dedicated tools like WebPageTest, and Calibre's inclusion of it within the monitoring dashboard is a genuine differentiator.
Multiple test locations
Calibre tests from multiple geographic regions, allowing teams to understand how CDN configuration and server latency affect users in different parts of the world. A site that performs well from a data centre in Virginia may perform poorly from Singapore if assets are not cached at an edge node nearby. By running tests from multiple locations in parallel, Calibre makes these geographic disparities visible in the same dashboard rather than requiring separate tooling per region. For global products, this is not a cosmetic feature — geographic latency is often the single largest driver of LCP variance between user segments.
Team collaboration and integrations
Calibre is designed for engineering organisations, not individual developers. The platform supports multiple team members with role-based access controls, shared dashboards, and team-wide alert configurations. Native Slack integration sends performance regression alerts to engineering channels rather than email inboxes, which is meaningfully more likely to result in a fast response. The Jira integration allows teams to create performance-related tickets directly from the Calibre interface, routing regressions into the existing issue tracking workflow without manual copying of data. These integrations reflect a deliberate philosophy: performance work should live inside existing engineering workflows, not in a separate tool that engineers have to remember to check.
Honest assessment
Strengths and limitations
What Calibre does well
Best-in-class CI/CD integration
The GitHub PR check workflow is genuinely one of the most useful implementations of performance budgeting available. Teams that adopt it typically see a measurable reduction in the frequency of performance regressions reaching production — not because the tool is magic, but because it places the performance signal in the exact moment when an engineer can act on it.
Granular, configurable budgets
Calibre offers per-metric, per-profile budget configuration with both absolute and relative thresholds. This flexibility allows teams to set different standards for mobile and desktop, for critical pages versus secondary pages, and to evolve budgets alongside the product without manual recalibration.
Strong developer experience
The product has been built by people who deeply understand the developer workflow. The CLI tooling, API, and integration surface are well-documented and work reliably. Engineers who evaluate Calibre consistently note that it feels like a product designed by engineers, for engineers.
Historical data and build correlation
The ability to correlate performance changes with specific deploys compresses investigation time dramatically. Calibre's historical retention and deploy annotation features make it possible to answer the question 'which deploy caused this?' in seconds.
Filmstrip and waterfall debugging
The visual debugging tools available in Calibre — filmstrip, waterfall, resource breakdown — are typically found only in dedicated profiling tools like WebPageTest. Having them embedded in a monitoring product removes a significant context-switching cost for engineers investigating regressions.
Where Calibre has gaps
No revenue impact modeling
Calibre reports metrics and regressions, but it does not connect those metrics to business outcomes. A 500ms increase in LCP does not come with an estimate of how many conversions that costs. For teams where performance competes for engineering time against other product work, this absence makes it harder to build the business case for prioritising a fix.
No competitor benchmarking
Calibre monitors your own pages. It has no capability to run the same tests against competitor URLs, so teams cannot see whether they are gaining or losing ground relative to their competitive set. Understanding that your LCP improved from 3.2s to 2.8s is useful; knowing that your main competitor went from 2.9s to 2.1s in the same period is more useful still.
Single-organisation focus
Calibre is designed for one organisation monitoring its own sites. It does not have concepts like multi-client dashboards, client-specific access controls, or organisation-level groupings that agencies managing multiple client sites need. Every client would require a separate Calibre account with separate billing.
No white-label reporting
There is no mechanism to produce a branded PDF report for a client. Agency use cases — where a performance report is a deliverable that a client reviews in a meeting — are not served by the product.
Pricing scales steeply
Calibre's pricing is based on the number of pages monitored and test profiles configured. A site with many pages and multiple test profiles per page (mobile + desktop, multiple locations) can reach significant monthly costs. Teams with large page inventories may find the economics challenging.
No real-user monitoring
Calibre is a synthetic monitoring tool only. It does not collect field data from real visitors. The gap between lab performance (what Calibre measures) and field performance (what users actually experience) can be substantial, especially for sites with highly variable user behaviour, many device types, or significant traffic from regions with poor connectivity.
| Capability | AuditJet | Calibre |
|---|---|---|
| Continuous synthetic monitoring | Hourly scans with regression alerts via Slack, email, and webhooks. | Scheduled Lighthouse tests with alerts and GitHub status checks. |
| Performance budgets | Score-based thresholds and trend alerts per metric with regression notifications. | Granular per-metric budgets with absolute and relative thresholds. Best-in-class. |
| PR-blocking CI/CD integration | Post-deploy monitoring; does not natively block PR merges. | GitHub App blocks PR merges on budget failures. Genuine strength. |
| Revenue impact modeling | Connects CWV score changes to estimated conversion and revenue loss. | Not available. |
| Competitor benchmarking | Monitor competitor pages on the same schedule as your own (Pro+). | Not available. |
| AI Fix Blueprints | Code-level fix guidance per issue, generated by AI (Watchdog+). | Standard Lighthouse recommendations only. |
| White-label reporting | Branded PDF reports for agencies (Agency plan). | Not available. |
| Multi-client dashboards | Dedicated agency tier with multi-client management. | Single-organisation only; separate account required per client. |
| Filmstrip & waterfall debugging | Not offered. | Visual filmstrip and resource waterfall included. Genuine strength. |
| Real-user monitoring (RUM) | Sub-1KB RUM script captures 75th-percentile LCP, INP, and CLS from real visitors (PRO+). | Not offered (synthetic monitoring only). |
Positioning
AuditJet vs Calibre: where they diverge
Calibre and AuditJet overlap significantly at the core monitoring loop: both run scheduled Lighthouse tests, both track metrics over time, and both alert when something changes. A team evaluating the two platforms should not expect to find a clear winner on the basics — both do the basics well. The meaningful differences emerge in the use cases each product is designed around.
Calibre is optimised for the engineering workflow. Its development has been driven by the question: "How do we make performance regressions visible to engineers at the exact moment they can fix them?" The answer Calibre arrived at was CI/CD integration — blocking PR merges on budget violations. This is a powerful solution to a specific problem. The teams that benefit most from it are engineering teams with a mature deployment pipeline, a culture of feature branches and pull requests, and preview deployments that Calibre can test before code reaches production. If that describes your team, Calibre's CI integration is genuinely hard to replicate elsewhere.
AuditJet is optimised for business impact visibility. Its development has been driven by a different question: "Why should a product manager or CFO care about a 300ms LCP regression?" The answer AuditJet provides is revenue modeling — connecting the performance change to an estimate of how many conversions it affects. This matters because the people who control engineering time — product managers, VPs of engineering, CFOs — are not typically motivated by Lighthouse scores in isolation. They are motivated by business outcomes. An alert that says "LCP increased by 300ms" competes in the sprint backlog against features with clear revenue stories. An alert that says "LCP increased by 300ms, estimated conversion impact: 1.2% decrease, estimated monthly revenue impact: $4,300" has a very different position in that conversation.
Choose Calibre if
- — Your team lives in GitHub and wants PR-level performance budget checks
- — You need waterfall and filmstrip debugging tools within the monitoring platform
- — You are monitoring a single organisation's pages
- — Your primary audience for performance data is engineers, not executives or clients
- — You want to catch regressions before they reach production rather than after
Choose AuditJet if
- — You need to show stakeholders what a performance regression costs in revenue
- — You are an agency managing multiple client sites and need white-label reporting
- — You want competitor benchmarking to understand your performance relative to the market
- — You want AI-generated fix guidance rather than generic Lighthouse recommendations
- — Your audience includes non-technical stakeholders who need business context, not raw metrics
For agencies, the difference is particularly stark. An agency billing performance monitoring as a service needs to produce client deliverables — branded reports, executive summaries, clear before-and-after comparisons. Calibre produces none of this. AuditJet's Agency plan is built specifically for this workflow: white-label PDF reports, multi-client dashboards, and branded output that an agency can send to a client and have reviewed in a quarterly business review. This is not a minor feature difference; it is a fundamentally different use case.
The "why fix it" problem is also worth dwelling on. Many engineering teams know their performance is poor. They have the Lighthouse scores; they know LCP is above 3 seconds. The blocker is not awareness — it is prioritisation. When every sprint is contested between new features and technical work, the performance issue loses unless someone can attach a number to it. AuditJet's revenue impact estimates provide that number. They are estimates, not guarantees, but they are grounded in the published research on the relationship between loading time and conversion rate, and they give performance engineers a tool that engineering managers and product managers can act on.
Decision guide
Who should use each platform?
Use Calibre if...
- —Your engineering team has a mature PR-based deployment workflow and wants performance budgets enforced at the point of code review.
- —You want visual debugging tools — filmstrip and waterfall charts — available within the monitoring platform without switching to WebPageTest.
- —Your audience for performance data is primarily developers and performance engineers who want precise, technical metrics.
- —You are monitoring a single organisation and do not need multi-client management or white-label outputs.
- —You value pre-production regression prevention over post-deploy revenue impact analysis.
Use AuditJet if...
- —You need performance data to be legible to non-technical stakeholders — connecting metric changes to revenue estimates rather than just displaying scores.
- —You are an agency managing multiple client sites and need white-label PDF reports and multi-client dashboards.
- —You want competitor benchmarking so you can see whether you are gaining or losing performance ground relative to competitors.
- —You want AI-generated, code-level fix guidance per issue rather than generic Lighthouse recommendations.
- —Sprint planning requires a business case to justify performance work, and you need numbers that can compete with feature requests.
Frequently asked questions
Calibre and AuditJet — common questions
What is a performance budget?
A performance budget is a set of limits you place on specific web performance metrics — for example, 'Largest Contentful Paint must not exceed 2.5 seconds' or 'total JavaScript bundle size must stay under 300 KB'. When a deploy or code change causes a metric to exceed its budget, the team is alerted immediately. Performance budgets transform abstract scores into enforceable engineering constraints, making performance a first-class concern in the development workflow rather than an afterthought addressed after launch.
How does Calibre's GitHub integration work?
Calibre integrates with GitHub via a GitHub App that runs performance tests as part of your CI/CD pipeline. When a pull request is opened, Calibre automatically triggers a synthetic Lighthouse test against a preview deployment and compares the results against the baseline and any configured budgets. If a budget is exceeded — say LCP increases by more than 200ms compared to the main branch — Calibre posts a failing status check on the PR, preventing merge until the regression is addressed or the budget is explicitly overridden.
Can AuditJet block deployments like Calibre?
AuditJet is primarily focused on continuous post-deploy monitoring with regression alerts rather than pre-merge PR blocking. Calibre's CI-gated PR checks are one of its genuine strengths for engineering teams that want to prevent regressions from ever reaching production. AuditJet catches regressions immediately after they deploy and connects them to estimated revenue impact, which is a complementary — rather than competing — capability.
What is the difference between synthetic and real-user monitoring?
Synthetic monitoring runs scripted tests from controlled lab environments on a schedule — the same URL, same device configuration, same network conditions every time. This produces reproducible, comparable results and can catch regressions before any real user encounters them. Real-user monitoring (RUM) collects performance data from actual visitors using the Web Performance APIs built into browsers, capturing the true distribution of experiences across all device types, networks, and geographies. Both have value: synthetic for regression detection, RUM for understanding real-world impact. Calibre uses synthetic monitoring only; it does not offer a RUM product.
Does Calibre offer real-user monitoring?
No. Calibre is a synthetic monitoring platform — it runs scheduled Lighthouse tests from controlled environments. It does not collect field data from real visitors. This means Calibre's metrics reflect lab conditions rather than the distribution of actual user experiences. For teams that need field data (Core Web Vitals as measured by real browsers), the Chrome UX Report (CrUX) dataset, Google Search Console, or a dedicated RUM tool would need to be used alongside Calibre.
How does AuditJet compare to Calibre on pricing?
Calibre's pricing scales with the number of pages and test profiles, starting around $149/month for a small site. AuditJet's Watchdog plan starts at $49/month and includes continuous monitoring with AI fix guidance. The Pro plan at $199/month adds competitor benchmarking and revenue impact estimates. For agencies managing multiple clients, AuditJet's Agency plan at $499/month includes white-label reporting and multi-client dashboards — a capability Calibre does not offer at any price point.
Which is better for agencies — AuditJet or Calibre?
AuditJet is the stronger choice for agencies. It offers a dedicated Agency plan with white-label PDF reports, multi-client dashboards, and branded reporting that can be sent directly to clients. Calibre is designed for single-organisation use and does not support multi-client management or white-label outputs. For an agency billing performance monitoring as a service, the ability to produce branded client-facing reports is a significant operational advantage.
Can I use both Calibre and AuditJet?
Yes, and some teams do. Calibre's CI-gated PR checks are genuinely useful for engineering teams that want to catch regressions before they reach production. AuditJet's revenue impact modeling and competitor benchmarking serve a different audience — product managers, executives, and agency clients who need to understand the business consequence of performance changes. A team could use Calibre for pre-merge budget enforcement and AuditJet for business-context dashboards and competitor tracking without significant overlap.
Performance monitoring that speaks the language of the business.
AuditJet monitors Core Web Vitals continuously, connects every regression to a revenue estimate, benchmarks against your competitors, and generates AI-powered fix guidance — all in a single dashboard that works for engineers and executives alike.