Extending Spectacles with API-Triggered Runs
We’re big fans of APIs at Spectacles—our core product is built on top of Looker’s API! We love the flexibility of APIs for customizing and tailoring tools to your exact needs. APIs are also great for integrating tools with orchestration solutions like Airflow, Dagster, or Prefect.
We already enable you to trigger Spectacles manually, on a schedule, or via pull and merge requests, but we wanted to support deeper integration with other layers of the data stack.
Starting today, you can trigger Spectacles runs or get information about existing runs via our REST API. If you want to dive right in, check out our API reference docs here.
Why would you want to do this?
Let’s say you have an Airflow or Prefect job that transforms data in your warehouse each morning. If that data eventually ends up in Looker, you might want to test your dimensions with Spectacles immediately after each transformation job finishes.
Because Spectacles runs lightweight SQL queries to test each LookML dimension, you’ll catch any SQL errors arising from changes in the underlying data.
Ever forget to use a NULLIF or SAFE_DIVIDE when dividing two numbers in a LookML dimension? Sometimes SQL errors in your dimensions crop up unexpectedly. If your ETL added a row with a zero in the denominator column, you’ll start getting a divide-by-zero error, not because your SQL is suddenly broken, but because of newly loaded data.
Using the Spectacles API, you can kick off Spectacles SQL validation as part of your transformation job, ensuring you catch issues like this right away.
We're giving a lot of thought to integrating Spectacles with the transformation layer. If you use dbt for transformations… stay tuned, we’ll be releasing a deeper integration with dbt in the coming weeks!
