Webhooks

Webhooks can be set up to receive notifications about changes to prompt templates. This functionality is particularly useful for storing prompts in cache, allowing for quicker retrieval without slowing down releases.

Setting Up Webhooks

To set up a webhook, go to the Webhook section in the Settings page. Enter the URL of the endpoint you want to send the webhook to and click Submit.

Event Payload Format

When an event occurs, we send a POST request with a payload in this structure:

{
  "event_type": "string",
  "details": "object",
  "user_id": "number",
  "user_name": "string or null",
  "user_email": "string or null",
  "workspace_id": "number",
  "timestamp": "ISO 8601 format timestamp",
}

Supported Event Types

Event Type	Description	Details
prompt_template_version_created	When a new version of a prompt template is created.	prompt_template_name (string) prompt_template_version_number (number) prompt_template_id (number)
prompt_template_name_changed	When a prompt template’s name is changed.	prompt_template_id (number) prompt_template_name (string) old_prompt_template_name (string)
prompt_template_deleted	When a prompt template is deleted.	prompt_template_id (number) prompt_template_name (string)
prompt_template_label_created	When a new release label for a prompt template is created.	prompt_template_id (number) prompt_template_name (string) prompt_template_version_number (number) prompt_template_label (string)
prompt_template_label_deleted	When a release label for a prompt template is deleted.	prompt_template_id (number) prompt_template_name (string) prompt_template_version_number (number) prompt_template_label (string)
prompt_template_label_moved	When a release label is moved between prompt template versions.	prompt_template_id (number) prompt_template_name (string) prompt_template_version_number (number) old_prompt_template_version_number (number) prompt_template_label (string)
prompt_template_updated	When a snippet imported in a prompt template is updated.	prompt_template_id (number) prompt_template_name (string) prompt_template_version_number (number)
agent_run_finished	When an agent (workflow) run is completed. Note: This event may fire multiple times for the same execution and is not triggered for runs from the dashboard, only when called via SDK or API.	agent_name (string) agent_id (number) agent_execution_id (number)
report_finished	When a evaluation report is completed.	report_id (number) report_name (string)
dataset_version_created_by_file	When a dataset version is successfully created from a file upload.	dataset_id (number) dataset_version_number (number)
dataset_version_created_by_file_failed	When dataset file processing fails.	error_message (string)
dataset_version_created_from_filter_params	When a dataset version is created from filter parameters.	dataset_id (number) rows_added (number) dataset_version_number (number)

Example Payload

{
  "event_type":"prompt_template_label_moved",
  "details":{
    "prompt_template_id":12,
    "prompt_template_name":"Hello",
    "prompt_template_version_number":9,
    "old_prompt_template_version_number":8,
    "prompt_template_label":"prod"
  },
  "user_id":1,
  "user_name":"John Doe",
  "user_email":"[e-mail]",
  "workspace_id":1,
  "timestamp":"2023-12-01T22:05:57.924833"
}

Securing Your Webhook

When you create a webhook, you’ll receive a webhook secret signature that looks like this:

This secret is used to verify that incoming webhook requests are authentic and come from PromptLayer. The signature is included in the X-PromptLayer-Signature header of each webhook request.

Verifying Webhook Signatures

Here are code examples showing how to verify the signatures:

Python

import hmac
import hashlib
import json

signature = "HEADER FROM X-PromptLayer-Signature" # Replace with actual header value
secret_key = "SECRET KEY FROM PROMPTLAYER DASHBOARD" # Replace with actual secret key
payload = {} # Replace with actual payload
payload_str = json.dumps(payload, sort_keys=True)
expected_signature = hmac.new(
    key=secret_key.encode(),
    msg=payload_str.encode('utf-8'),
    digestmod=hashlib.sha256
).hexdigest()

if hmac.compare_digest(expected_signature, signature):
    print("Signature is valid")
else:
    print("Signature is invalid")

Using Webhooks for Caching

Webhooks are particularly useful for maintaining a local cache of prompts, eliminating the need for extra round-trips to PromptLayer. Here’s how it works:

Implementation Example

1. Handle webhook events:

# pseudocode
def handle_pl_webhook(event):
    prompt = event["data"]
    db.prompts.upsert(prompt["prompt_template_name"], prompt)

2. Serve traffic from cache:

prompt = db.prompts.get("order-summary")
llm_response = openai.chat.completions.create(...)
queue.enqueue(track_to_promptlayer, llm_response)

Tip: Most teams push the track_to_promptlayer onto a Redis or SQS queue so as to not block on the logging of a request.